2024-01-19T04:45:24.782 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-19T04:45:24.782 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038 --verbose --owner scheduled_yuriw@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm} --name yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/orig.config.yaml 2024-01-19T04:45:24.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-19T04:45:24.906 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038 branch: wip-yuri8-testing-2024-01-18-0823 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm} email: yweinste@redhat.com first_in_suite: false job_id: '7521038' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri8-testing-2024-01-18-0823 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 0edf41a622739843d6b978b179ff3227b476dd9d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0edf41a622739843d6b978b179ff3227b476dd9d subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri8-testing-2024-01-18-0823 sha1: 0edf41a622739843d6b978b179ff3227b476dd9d owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6631 sha1: 0edf41a622739843d6b978b179ff3227b476dd9d sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri8-testing-2024-01-18-0823 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 0edf41a622739843d6b978b179ff3227b476dd9d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfCsrSuIklKEIe2LJNcxUOdkdiyQIl89QsL9zj3TrLRIJqWDvPNGcqZ4/uam4JLHEob6pP0RSQxbCMD2dg06FQ= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNazwqQDaviTxfMbB+N/FPtO/PTb6ysR06SdlXD5H6640Cm2XYEZznkQmi/ZEo90JW1vpWugaybplenJ7ZsRFEw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-18_21:19:14 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-19T04:45:24.907 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa; will attempt to use it 2024-01-19T04:45:24.907 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa/tasks 2024-01-19T04:45:24.907 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-19T04:45:24.909 INFO:teuthology.task.internal:Checking packages... 2024-01-19T04:45:24.931 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0edf41a622739843d6b978b179ff3227b476dd9d' 2024-01-19T04:45:24.932 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-19T04:45:24.932 INFO:teuthology.packaging:ref: None 2024-01-19T04:45:24.932 INFO:teuthology.packaging:tag: None 2024-01-19T04:45:24.932 INFO:teuthology.packaging:branch: wip-yuri8-testing-2024-01-18-0823 2024-01-19T04:45:24.932 INFO:teuthology.packaging:sha1: 0edf41a622739843d6b978b179ff3227b476dd9d 2024-01-19T04:45:24.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri8-testing-2024-01-18-0823 2024-01-19T04:45:25.085 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-766.g0edf41a6 2024-01-19T04:45:25.087 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-19T04:45:25.518 INFO:teuthology.task.internal:no buildpackages task found 2024-01-19T04:45:25.518 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-19T04:45:25.719 INFO:teuthology.task.internal:Saving configuration 2024-01-19T04:45:25.753 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-19T04:45:25.760 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-19T04:45:25.785 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-19 04:40:25.451500', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfCsrSuIklKEIe2LJNcxUOdkdiyQIl89QsL9zj3TrLRIJqWDvPNGcqZ4/uam4JLHEob6pP0RSQxbCMD2dg06FQ='} 2024-01-19T04:45:25.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-19 04:40:25.450179', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNazwqQDaviTxfMbB+N/FPtO/PTb6ysR06SdlXD5H6640Cm2XYEZznkQmi/ZEo90JW1vpWugaybplenJ7ZsRFEw='} 2024-01-19T04:45:25.805 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-19T04:45:25.813 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-19T04:45:25.813 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-19T04:45:25.813 INFO:teuthology.run_tasks:Running task console_log... 2024-01-19T04:45:25.883 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f722f6e3160>, signals=[15]) 2024-01-19T04:45:25.884 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-19T04:45:25.890 INFO:teuthology.task.internal:Opening connections... 2024-01-19T04:45:25.890 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:45:25.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:45:26.002 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:45:26.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:45:26.090 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-19T04:45:26.101 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-01-19T04:45:26.119 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-01-19T04:45:26.119 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-01-19T04:45:26.176 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-01-19T04:45:26.177 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-19T04:45:26.177 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-01-19T04:45:26.177 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-19T04:45:26.177 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-19T04:45:26.177 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-19T04:45:26.177 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-01-19T04:45:26.198 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-01-19T04:45:26.216 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-01-19T04:45:26.216 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-01-19T04:45:26.271 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-01-19T04:45:26.272 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-19T04:45:26.272 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-01-19T04:45:26.272 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-19T04:45:26.272 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-19T04:45:26.272 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-19T04:45:26.272 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-01-19T04:45:26.298 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-19T04:45:26.313 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-19T04:45:26.320 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-19T04:45:26.320 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-01-19T04:45:26.323 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-01-19T04:45:26.340 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-19T04:45:26.348 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-19T04:45:26.348 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-01-19T04:45:26.380 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-01-19T04:45:26.481 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-19T04:45:26.536 INFO:teuthology.run_tasks:Running task kernel... 2024-01-19T04:45:26.553 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-01-19T04:45:26.553 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-01-19T04:45:26.553 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-19T04:45:26.554 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-19T04:45:26.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:45:26.571 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-01-19T04:45:26.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:45:26.572 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-01-19T04:45:26.628 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-361.el9.x86_64 2024-01-19T04:45:26.628 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.14.0-361.el9.x86_64 2024-01-19T04:45:26.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-01-19T04:45:26.629 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-361.el9.x86_64 2024-01-19T04:45:26.630 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-361.el9.x86_64 2024-01-19T04:45:26.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-01-19T04:45:28.765 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2024-01-19T04:45:28.818 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2024-01-19T04:45:30.466 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2024-01-19T04:45:30.535 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-01-19T04:45:32.626 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-01-19T04:45:33.139 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-01-19T04:45:34.918 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-01-19T04:45:35.349 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-01-19T04:45:40.235 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 5.9 MB 00:01 2024-01-19T04:45:40.783 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.9 MB 00:01 2024-01-19T04:45:42.070 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 15 kB 00:00 2024-01-19T04:45:42.175 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 15 kB 00:00 2024-01-19T04:45:43.548 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2024-01-19T04:45:44.312 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:02 2024-01-19T04:45:48.412 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-01-19T04:45:49.293 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-01-19T04:45:50.652 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-19T04:45:50.653 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-19T04:45:50.746 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout: kernel x86_64 5.14.0-408.el9 CentOS-BaseOS 6.0 M 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core x86_64 5.14.0-408.el9 CentOS-BaseOS 21 M 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules x86_64 5.14.0-408.el9 CentOS-BaseOS 39 M 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core x86_64 5.14.0-408.el9 CentOS-BaseOS 33 M 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:45:50.758 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-19T04:45:50.759 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-19T04:45:50.759 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-01-19T04:45:50.759 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:45:50.759 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 99 M 2024-01-19T04:45:50.759 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 122 M 2024-01-19T04:45:50.759 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-19T04:45:51.305 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): kernel-5.14.0-408.el9.x86_64.rpm 11 MB/s | 6.0 MB 00:00 2024-01-19T04:45:51.583 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-19T04:45:51.583 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-19T04:45:51.680 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-19T04:45:51.698 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-19T04:45:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-19T04:45:51.698 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-19T04:45:51.698 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-19T04:45:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-408.el9 CentOS-BaseOS 6.0 M 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-408.el9 CentOS-BaseOS 21 M 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-408.el9 CentOS-BaseOS 39 M 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-408.el9 CentOS-BaseOS 33 M 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-01-19T04:45:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:45:51.700 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 99 M 2024-01-19T04:45:51.700 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 122 M 2024-01-19T04:45:51.700 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-19T04:45:51.964 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): kernel-core-5.14.0-408.el9.x86_64.rpm 17 MB/s | 21 MB 00:01 2024-01-19T04:45:52.102 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-408.el9.x86_64.rpm 15 MB/s | 6.0 MB 00:00 2024-01-19T04:45:52.380 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): kernel-modules-core-5.14.0-408.el9.x86_6 31 MB/s | 33 MB 00:01 2024-01-19T04:45:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-408.el9.x86_64.rpm 21 MB/s | 21 MB 00:00 2024-01-19T04:45:53.164 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): kernel-modules-5.14.0-408.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-01-19T04:45:53.167 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-19T04:45:53.167 INFO:teuthology.orchestra.run.smithi137.stdout:Total 41 MB/s | 99 MB 00:02 2024-01-19T04:45:53.168 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-19T04:45:53.352 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-core-5.14.0-408.el9.x86_6 27 MB/s | 33 MB 00:01 2024-01-19T04:45:53.540 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-19T04:45:53.540 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-19T04:45:53.819 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-5.14.0-408.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-01-19T04:45:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-19T04:45:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:Total 47 MB/s | 99 MB 00:02 2024-01-19T04:45:53.823 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-19T04:45:54.280 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-19T04:45:54.281 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-19T04:45:54.588 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-19T04:45:54.588 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-19T04:45:55.326 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-19T04:45:55.326 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-19T04:45:56.976 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-19T04:45:57.643 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-core-5.14.0-408.el9.x86_64 1/4 2024-01-19T04:45:57.772 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-core-5.14.0-408.el9.x86_64 2/4 2024-01-19T04:45:57.865 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-19T04:45:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-408.el9.x86_64 1/4 2024-01-19T04:45:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-408.el9.x86_64 2/4 2024-01-19T04:46:00.515 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-408.el9.x86_64 2/4 2024-01-19T04:46:00.691 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-5.14.0-408.el9.x86_64 3/4 2024-01-19T04:46:01.640 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-408.el9.x86_64 2/4 2024-01-19T04:46:01.821 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-408.el9.x86_64 3/4 2024-01-19T04:46:05.358 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-408.el9.x86_64 3/4 2024-01-19T04:46:05.461 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-408.el9.x86_64 3/4 2024-01-19T04:46:07.000 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-core-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:39.429 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:39.429 INFO:teuthology.orchestra.run.smithi137.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-19T04:46:39.429 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:46:39.499 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:39.881 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:39.881 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-5.14.0-408.el9.x86_64 1/4 2024-01-19T04:46:39.881 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-core-5.14.0-408.el9.x86_64 2/4 2024-01-19T04:46:39.881 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-5.14.0-408.el9.x86_64 3/4 2024-01-19T04:46:40.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-core-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:40.358 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:46:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-19T04:46:40.359 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-5.14.0-408.el9.x86_64 2024-01-19T04:46:40.359 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core-5.14.0-408.el9.x86_64 2024-01-19T04:46:40.359 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-5.14.0-408.el9.x86_64 2024-01-19T04:46:40.359 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core-5.14.0-408.el9.x86_64 2024-01-19T04:46:40.359 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:46:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-19T04:46:40.903 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-01-19T04:46:41.024 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-408.el9.x86_64 2024-01-19T04:46:41.024 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-408.el9.x86_64 2024-01-19T04:46:41.024 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-408.el9.x86_64 2024-01-19T04:46:41.024 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-19T04:46:41.024 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-19T04:46:41.024 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-01-19T04:46:41.025 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-01-19T04:46:41.025 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-01-19T04:46:41.132 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-408.el9.x86_64 2024-01-19T04:46:41.132 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-408.el9.x86_64 2024-01-19T04:46:41.132 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-01-19T04:46:41.165 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-01-19T04:46:41.165 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-01-19T04:46:41.165 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.06 2024-01-19T04:46:41.165 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 68.el9 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Group : Unspecified 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 8264254 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : builder@centos.org 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-01-19T04:46:41.166 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-01-19T04:46:41.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:46:41.167 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-19T04:46:41.167 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-19T04:46:41.167 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-19T04:46:41.167 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-01-19T04:46:41.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:46:41.167 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-01-19T04:46:41.168 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-19T04:46:41.168 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 5.14.0-408.el9.x86_64 2024-01-19T04:46:41.168 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-19T04:46:41.769 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-01-19T04:46:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-19T04:46:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:46:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:43.160 INFO:teuthology.orchestra.run.smithi137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-19T04:46:43.192 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-01-19T04:46:43.194 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-01-19T04:46:43.209 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.njgX2j0vL8 2024-01-19T04:46:43.209 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.njgX2j0vL8 2024-01-19T04:46:43.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.njgX2j0vL8 2024-01-19T04:46:43.412 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.njgX2j0vL8 is 6KB 2024-01-19T04:46:43.460 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.njgX2j0vL8 2024-01-19T04:46:43.474 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-01-19T04:46:43.539 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-19T04:46:43.539 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-19T04:46:43.539 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-19T04:46:43.539 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-408.el9.x86_64.conf 2024-01-19T04:46:43.541 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-408.el9.x86_64 2024-01-19T04:46:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-408.el9.x86_64 1/4 2024-01-19T04:46:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-408.el9.x86_64 2/4 2024-01-19T04:46:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-408.el9.x86_64 3/4 2024-01-19T04:46:43.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-408.el9.x86_64 4/4 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-408.el9.x86_64 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-408.el9.x86_64 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-408.el9.x86_64 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-408.el9.x86_64 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:46:44.029 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-19T04:46:44.574 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-01-19T04:46:44.705 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-408.el9.x86_64 2024-01-19T04:46:44.705 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-408.el9.x86_64 2024-01-19T04:46:44.706 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-408.el9.x86_64 2024-01-19T04:46:44.706 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-19T04:46:44.706 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-19T04:46:44.706 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-01-19T04:46:44.706 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-01-19T04:46:44.706 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-01-19T04:46:44.818 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-408.el9.x86_64 2024-01-19T04:46:44.819 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-408.el9.x86_64 2024-01-19T04:46:44.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-01-19T04:46:44.862 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-01-19T04:46:44.862 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-01-19T04:46:44.862 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-01-19T04:46:44.862 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 68.el9 2024-01-19T04:46:44.863 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-01-19T04:46:44.863 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-19T04:46:44.863 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-01-19T04:46:44.863 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8264254 2024-01-19T04:46:44.863 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-01-19T04:46:44.863 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-19T04:46:44.864 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-19T04:46:44.864 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-19T04:46:44.864 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-19T04:46:44.864 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-01-19T04:46:44.864 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-01-19T04:46:44.864 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-19T04:46:44.865 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-01-19T04:46:44.865 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-01-19T04:46:44.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:46:44.865 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-19T04:46:44.865 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-19T04:46:44.865 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-19T04:46:44.865 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-01-19T04:46:44.866 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:46:44.866 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-01-19T04:46:44.867 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-19T04:46:44.867 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-408.el9.x86_64 2024-01-19T04:46:44.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-19T04:46:45.446 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-01-19T04:46:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-19T04:46:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-01-19T04:46:46.853 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-01-19T04:46:46.868 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.q53QgleR9O 2024-01-19T04:46:46.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q53QgleR9O 2024-01-19T04:46:46.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.q53QgleR9O 2024-01-19T04:46:47.087 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.q53QgleR9O is 6KB 2024-01-19T04:46:47.135 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.q53QgleR9O 2024-01-19T04:46:47.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-01-19T04:46:47.218 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-19T04:46:47.219 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-19T04:46:47.219 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-19T04:46:47.219 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-5.14.0-408.el9.x86_64.conf 2024-01-19T04:46:47.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-408.el9.x86_64 2024-01-19T04:46:47.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-01-19T04:46:47.475 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-01-19T04:47:13.722 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-01-19T04:47:13.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-19T04:47:13.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:47:17.389 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-01-19T04:47:17.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-19T04:47:17.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:47:32.156 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-01-19T04:47:35.740 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-01-19T04:47:41.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-19T04:47:41.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:47:44.220 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-01-19T04:47:44.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-19T04:47:44.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:47:56.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-19T04:47:56.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:48:03.132 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-01-19T04:48:15.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-19T04:48:15.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:48:15.893 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-01-19T04:48:16.291 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-19T04:48:16.291 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-408.el9.x86_64"... 2024-01-19T04:48:16.291 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-01-19T04:48:16.305 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-408.el9.x86_64 2024-01-19T04:48:16.305 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-408.el9.x86_64 vs 5.14.0-408.el9.x86_64 2024-01-19T04:48:16.305 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-19T04:48:16.305 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-19T04:48:17.306 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-01-19T04:48:17.307 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-19T04:48:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-01-19T04:48:17.399 DEBUG:teuthology.parallel:result is None 2024-01-19T04:48:28.004 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-01-19T04:48:28.535 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-19T04:48:28.535 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-408.el9.x86_64"... 2024-01-19T04:48:28.535 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-01-19T04:48:28.549 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-408.el9.x86_64 2024-01-19T04:48:28.549 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-408.el9.x86_64 vs 5.14.0-408.el9.x86_64 2024-01-19T04:48:28.549 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-19T04:48:28.549 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-19T04:48:29.550 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-01-19T04:48:29.550 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-19T04:48:29.641 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-01-19T04:48:29.686 DEBUG:teuthology.parallel:result is None 2024-01-19T04:48:29.686 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-19T04:48:29.694 INFO:teuthology.task.internal:Creating test directory... 2024-01-19T04:48:29.695 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-19T04:48:29.698 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-19T04:48:29.721 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-19T04:48:29.728 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-19T04:48:29.735 INFO:teuthology.task.internal:Creating archive directory... 2024-01-19T04:48:29.735 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-19T04:48:29.756 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-19T04:48:29.809 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-19T04:48:29.816 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-19T04:48:29.816 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-19T04:48:29.825 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-19T04:48:29.856 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-19T04:48:29.870 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-19T04:48:29.906 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-19T04:48:29.926 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-19T04:48:29.928 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-19T04:48:29.935 INFO:teuthology.task.internal:Configuring sudo... 2024-01-19T04:48:29.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-19T04:48:29.938 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-19T04:48:29.982 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-19T04:48:29.991 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-19T04:48:29.991 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-19T04:48:30.008 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-19T04:48:30.041 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-19T04:48:30.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-19T04:48:30.155 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-19T04:48:30.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-19T04:48:30.282 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:48:30.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-19T04:48:30.351 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-19T04:48:30.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-19T04:48:30.469 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-19T04:48:30.511 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-19T04:48:30.553 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:48:30.553 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-19T04:48:30.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-01-19T04:48:30.623 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-01-19T04:48:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-19T04:48:30.710 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-19T04:48:31.038 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-19T04:48:31.046 INFO:teuthology.task.internal:Starting timer... 2024-01-19T04:48:31.046 INFO:teuthology.run_tasks:Running task pcp... 2024-01-19T04:48:31.060 INFO:teuthology.run_tasks:Running task selinux... 2024-01-19T04:48:31.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-01-19T04:48:31.143 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-01-19T04:48:31.145 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-01-19T04:48:31.217 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-01-19T04:48:31.218 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-19T04:48:31.218 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-01-19T04:48:31.241 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-01-19T04:48:31.242 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-01-19T04:48:31.272 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-01-19T04:48:31.273 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2024-01-19T04:48:31.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-19T04:48:31.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:48:31.310 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-19T04:48:31.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:48:31.340 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-19T04:48:31.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-01-19T04:48:31.385 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-01-19T04:48:31.416 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-19T04:48:31.426 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-19T04:48:31.446 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-19T04:48:31.447 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-19T04:52:32.214 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2024-01-19T04:52:32.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-19T04:52:32.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:52:32.318 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-01-19T04:52:32.400 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-19T04:52:32.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-19T04:52:32.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-19T04:52:32.488 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-01-19T04:52:32.567 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-19T04:52:32.567 INFO:teuthology.run_tasks:Running task clock... 2024-01-19T04:52:32.580 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-19T04:52:32.580 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-19T04:52:32.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-19T04:52:32.583 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-19T04:52:32.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-19T04:52:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-19T04:52:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-19T04:52:32.637 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-19T04:52:32.654 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-19T04:52:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-01-19T04:52:32.675 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-01-19T04:52:32.684 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-01-19T04:52:32.692 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-19T04:52:32.697 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-01-19T04:52:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-19T04:52:32.714 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-19T04:52:32.730 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-19T04:52:32.773 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-01-19T04:52:32.776 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-19T04:52:32.776 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-01-19T04:52:32.776 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.777 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.777 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.777 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.784 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-01-19T04:52:32.788 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-19T04:52:32.788 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-01-19T04:52:32.788 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.788 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.788 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.788 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:52:32.789 INFO:teuthology.run_tasks:Running task install... 2024-01-19T04:52:32.796 DEBUG:teuthology.task.install:project ceph 2024-01-19T04:52:32.797 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d'}} 2024-01-19T04:52:32.797 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d'} 2024-01-19T04:52:32.797 INFO:teuthology.task.install:Using flavor: default 2024-01-19T04:52:32.803 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-19T04:52:32.803 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-01-19T04:52:32.803 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d', 'tag': None, 'wait_for_package': False} 2024-01-19T04:52:32.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0edf41a622739843d6b978b179ff3227b476dd9d 2024-01-19T04:52:32.805 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d', 'tag': None, 'wait_for_package': False} 2024-01-19T04:52:32.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0edf41a622739843d6b978b179ff3227b476dd9d 2024-01-19T04:52:32.963 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/ 2024-01-19T04:52:32.963 INFO:teuthology.task.install.rpm:Package version is 19.0.0-766.g0edf41a6 2024-01-19T04:52:32.976 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/ 2024-01-19T04:52:32.976 INFO:teuthology.task.install.rpm:Package version is 19.0.0-766.g0edf41a6 2024-01-19T04:52:33.204 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-19T04:52:33.204 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:52:33.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-19T04:52:33.231 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-01-18-0823/0edf41a622739843d6b978b179ff3227b476dd9d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-19T04:52:33.231 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:52:33.232 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-19T04:52:33.233 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-19T04:52:33.233 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0edf41a622739843d6b978b179ff3227b476dd9d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-19T04:52:33.261 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-19T04:52:33.261 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0edf41a622739843d6b978b179ff3227b476dd9d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-19T04:52:33.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-19T04:52:33.329 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-19T04:52:33.381 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-19T04:52:33.409 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-19T04:52:33.447 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-01-19T04:52:33.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-01-19T04:52:33.476 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-01-19T04:52:33.477 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-01-19T04:52:33.709 INFO:teuthology.orchestra.run.smithi031.stdout:73 files removed 2024-01-19T04:52:33.718 INFO:teuthology.orchestra.run.smithi137.stdout:73 files removed 2024-01-19T04:52:33.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-19T04:52:33.745 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-19T04:52:34.394 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 244 kB/s | 80 kB 00:00 2024-01-19T04:52:34.406 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 231 kB/s | 80 kB 00:00 2024-01-19T04:52:34.728 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 33 kB/s | 9.5 kB 00:00 2024-01-19T04:52:34.765 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 31 kB/s | 9.5 kB 00:00 2024-01-19T04:52:35.117 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-01-19T04:52:35.137 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-01-19T04:52:35.720 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2024-01-19T04:52:36.135 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 6.7 MB/s | 6.4 MB 00:00 2024-01-19T04:52:37.177 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-01-19T04:52:37.598 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-01-19T04:52:38.326 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-01-19T04:52:39.214 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.0 MB 00:01 2024-01-19T04:52:41.406 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-01-19T04:52:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-01-19T04:52:46.124 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 5.9 MB 00:01 2024-01-19T04:52:47.849 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 15 kB 00:00 2024-01-19T04:52:48.469 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 5.9 MB 00:01 2024-01-19T04:52:49.337 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2024-01-19T04:52:50.128 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-01-19T04:52:50.746 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 41 MB/s | 20 MB 00:00 2024-01-19T04:52:54.259 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-01-19T04:52:55.838 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-01-19T04:52:57.037 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-19T04:52:57.038 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-19T04:52:57.050 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-19T04:52:57.051 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-19T04:52:57.052 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-19T04:52:57.053 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-19T04:52:57.054 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-19T04:52:57.057 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-19T04:52:57.058 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-19T04:52:57.134 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-19T04:52:57.149 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================== 2024-01-19T04:52:57.149 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-19T04:52:57.149 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================== 2024-01-19T04:52:57.149 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-19T04:52:57.149 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-19T04:52:57.149 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 6.5 k 2024-01-19T04:52:57.149 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 5.1 M 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 886 k 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 145 k 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 1.5 M 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 133 k 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 1.7 M 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 7.4 M 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 51 k 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 8.6 M 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 46 M 2024-01-19T04:52:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 262 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 742 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 217 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 27 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 18 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 31 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 742 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 127 k 2024-01-19T04:52:57.151 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 161 k 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 321 k 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 298 k 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 99 k 2024-01-19T04:52:57.152 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 85 k 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 3.1 M 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 170 k 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-19T04:52:57.153 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 3.4 M 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 3.1 M 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-19T04:52:57.154 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 19 M 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 28 k 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 2.1 M 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 245 k 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 4.7 M 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 17 M 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 15 k 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 25 k 2024-01-19T04:52:57.155 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 164 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-19T04:52:57.156 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 499 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 4.8 M 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-19T04:52:57.157 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 45 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 130 k 2024-01-19T04:52:57.158 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-19T04:52:57.159 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-19T04:52:57.160 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-19T04:52:57.161 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-19T04:52:57.162 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================== 2024-01-19T04:52:57.163 INFO:teuthology.orchestra.run.smithi137.stdout:Install 139 Packages 2024-01-19T04:52:57.163 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 16 Packages 2024-01-19T04:52:57.163 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:52:57.163 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 209 M 2024-01-19T04:52:57.163 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-19T04:52:57.844 INFO:teuthology.orchestra.run.smithi137.stdout:(1/155): ceph-19.0.0-766.g0edf41a6.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2024-01-19T04:52:58.102 INFO:teuthology.orchestra.run.smithi137.stdout:(2/155): ceph-base-19.0.0-766.g0edf41a6.el9.x86 9.9 MB/s | 5.1 MB 00:00 2024-01-19T04:52:58.152 INFO:teuthology.orchestra.run.smithi137.stdout:(3/155): ceph-immutable-object-cache-19.0.0-766 2.8 MB/s | 145 kB 00:00 2024-01-19T04:52:58.303 INFO:teuthology.orchestra.run.smithi137.stdout:(4/155): ceph-mds-19.0.0-766.g0edf41a6.el9.x86_ 14 MB/s | 2.1 MB 00:00 2024-01-19T04:52:58.328 INFO:teuthology.orchestra.run.smithi137.stdout:(5/155): ceph-fuse-19.0.0-766.g0edf41a6.el9.x86 1.8 MB/s | 886 kB 00:00 2024-01-19T04:52:58.412 INFO:teuthology.orchestra.run.smithi137.stdout:(6/155): ceph-mgr-19.0.0-766.g0edf41a6.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-01-19T04:52:58.469 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-19T04:52:58.470 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-19T04:52:58.486 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-19T04:52:58.487 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-19T04:52:58.488 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-19T04:52:58.489 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-19T04:52:58.491 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-19T04:52:58.494 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-19T04:52:58.495 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-19T04:52:58.579 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-19T04:52:58.597 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================== 2024-01-19T04:52:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-19T04:52:58.597 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================== 2024-01-19T04:52:58.597 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-19T04:52:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-19T04:52:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 6.5 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 5.1 M 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 886 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 145 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 1.5 M 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 133 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 1.7 M 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 7.4 M 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 51 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 8.6 M 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 46 M 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 262 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 742 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 217 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 27 k 2024-01-19T04:52:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 18 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 31 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 742 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 127 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 161 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 321 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 298 k 2024-01-19T04:52:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 99 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 85 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 3.1 M 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 170 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-19T04:52:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 3.4 M 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 3.1 M 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 19 M 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 28 k 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 2.1 M 2024-01-19T04:52:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 245 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 4.7 M 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 17 M 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.0.0-766.g0edf41a6.el9 ceph-noarch 15 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 25 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 164 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-19T04:52:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 499 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 4.8 M 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-19T04:52:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 45 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.0.0-766.g0edf41a6.el9 ceph 130 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-19T04:52:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-19T04:52:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-19T04:52:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-19T04:52:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-19T04:52:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================== 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout:Install 139 Packages 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 16 Packages 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 209 M 2024-01-19T04:52:58.609 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-19T04:52:59.222 INFO:teuthology.orchestra.run.smithi031.stdout:(1/155): ceph-19.0.0-766.g0edf41a6.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2024-01-19T04:52:59.353 INFO:teuthology.orchestra.run.smithi137.stdout:(7/155): ceph-osd-19.0.0-766.g0edf41a6.el9.x86_ 18 MB/s | 17 MB 00:00 2024-01-19T04:52:59.639 INFO:teuthology.orchestra.run.smithi031.stdout:(2/155): ceph-fuse-19.0.0-766.g0edf41a6.el9.x86 2.1 MB/s | 886 kB 00:00 2024-01-19T04:52:59.739 INFO:teuthology.orchestra.run.smithi031.stdout:(3/155): ceph-immutable-object-cache-19.0.0-766 1.4 MB/s | 145 kB 00:00 2024-01-19T04:52:59.853 INFO:teuthology.orchestra.run.smithi137.stdout:(8/155): ceph-mon-19.0.0-766.g0edf41a6.el9.x86_ 3.1 MB/s | 4.7 MB 00:01 2024-01-19T04:52:59.954 INFO:teuthology.orchestra.run.smithi137.stdout:(9/155): ceph-selinux-19.0.0-766.g0edf41a6.el9. 251 kB/s | 25 kB 00:00 2024-01-19T04:53:00.123 INFO:teuthology.orchestra.run.smithi031.stdout:(4/155): ceph-base-19.0.0-766.g0edf41a6.el9.x86 4.4 MB/s | 5.1 MB 00:01 2024-01-19T04:53:00.398 INFO:teuthology.orchestra.run.smithi031.stdout:(5/155): ceph-mgr-19.0.0-766.g0edf41a6.el9.x86_ 5.4 MB/s | 1.5 MB 00:00 2024-01-19T04:53:00.457 INFO:teuthology.orchestra.run.smithi031.stdout:(6/155): ceph-mds-19.0.0-766.g0edf41a6.el9.x86_ 3.0 MB/s | 2.1 MB 00:00 2024-01-19T04:53:00.562 INFO:teuthology.orchestra.run.smithi137.stdout:(10/155): ceph-radosgw-19.0.0-766.g0edf41a6.el9 7.1 MB/s | 8.6 MB 00:01 2024-01-19T04:53:00.654 INFO:teuthology.orchestra.run.smithi137.stdout:(11/155): cephfs-mirror-19.0.0-766.g0edf41a6.el 2.3 MB/s | 217 kB 00:00 2024-01-19T04:53:00.746 INFO:teuthology.orchestra.run.smithi137.stdout:(12/155): cephfs-shell-19.0.0-766.g0edf41a6.el9 295 kB/s | 27 kB 00:00 2024-01-19T04:53:00.830 INFO:teuthology.orchestra.run.smithi137.stdout:(13/155): libcephfs-devel-19.0.0-766.g0edf41a6. 373 kB/s | 31 kB 00:00 2024-01-19T04:53:00.865 INFO:teuthology.orchestra.run.smithi031.stdout:(7/155): ceph-mon-19.0.0-766.g0edf41a6.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-01-19T04:53:01.313 INFO:teuthology.orchestra.run.smithi137.stdout:(14/155): libcephfs2-19.0.0-766.g0edf41a6.el9.x 1.5 MB/s | 742 kB 00:00 2024-01-19T04:53:01.414 INFO:teuthology.orchestra.run.smithi137.stdout:(15/155): libcephsqlite-19.0.0-766.g0edf41a6.el 1.6 MB/s | 164 kB 00:00 2024-01-19T04:53:01.530 INFO:teuthology.orchestra.run.smithi137.stdout:(16/155): librados-devel-19.0.0-766.g0edf41a6.e 1.1 MB/s | 127 kB 00:00 2024-01-19T04:53:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:(17/155): libradosstriper1-19.0.0-766.g0edf41a6 1.4 MB/s | 499 kB 00:00 2024-01-19T04:53:01.890 INFO:teuthology.orchestra.run.smithi031.stdout:(8/155): ceph-radosgw-19.0.0-766.g0edf41a6.el9. 8.4 MB/s | 8.6 MB 00:01 2024-01-19T04:53:01.965 INFO:teuthology.orchestra.run.smithi031.stdout:(9/155): ceph-selinux-19.0.0-766.g0edf41a6.el9. 335 kB/s | 25 kB 00:00 2024-01-19T04:53:02.648 INFO:teuthology.orchestra.run.smithi137.stdout:(18/155): ceph-common-19.0.0-766.g0edf41a6.el9. 3.8 MB/s | 19 MB 00:05 2024-01-19T04:53:02.748 INFO:teuthology.orchestra.run.smithi137.stdout:(19/155): python3-ceph-argparse-19.0.0-766.g0ed 453 kB/s | 45 kB 00:00 2024-01-19T04:53:02.840 INFO:teuthology.orchestra.run.smithi137.stdout:(20/155): python3-ceph-common-19.0.0-766.g0edf4 1.4 MB/s | 130 kB 00:00 2024-01-19T04:53:02.932 INFO:teuthology.orchestra.run.smithi137.stdout:(21/155): python3-cephfs-19.0.0-766.g0edf41a6.e 1.7 MB/s | 161 kB 00:00 2024-01-19T04:53:02.999 INFO:teuthology.orchestra.run.smithi137.stdout:(22/155): python3-rados-19.0.0-766.g0edf41a6.el 4.7 MB/s | 321 kB 00:00 2024-01-19T04:53:03.066 INFO:teuthology.orchestra.run.smithi031.stdout:(10/155): ceph-osd-19.0.0-766.g0edf41a6.el9.x86 6.4 MB/s | 17 MB 00:02 2024-01-19T04:53:03.066 INFO:teuthology.orchestra.run.smithi137.stdout:(23/155): python3-rbd-19.0.0-766.g0edf41a6.el9. 4.4 MB/s | 298 kB 00:00 2024-01-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi031.stdout:(11/155): cephfs-mirror-19.0.0-766.g0edf41a6.el 2.3 MB/s | 217 kB 00:00 2024-01-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi137.stdout:(24/155): python3-rgw-19.0.0-766.g0edf41a6.el9. 1.1 MB/s | 99 kB 00:00 2024-01-19T04:53:03.235 INFO:teuthology.orchestra.run.smithi137.stdout:(25/155): rbd-fuse-19.0.0-766.g0edf41a6.el9.x86 1.1 MB/s | 85 kB 00:00 2024-01-19T04:53:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:(26/155): librgw2-19.0.0-766.g0edf41a6.el9.x86_ 3.2 MB/s | 4.8 MB 00:01 2024-01-19T04:53:03.358 INFO:teuthology.orchestra.run.smithi031.stdout:(12/155): ceph-common-19.0.0-766.g0edf41a6.el9. 4.4 MB/s | 19 MB 00:04 2024-01-19T04:53:03.383 INFO:teuthology.orchestra.run.smithi031.stdout:(13/155): cephfs-shell-19.0.0-766.g0edf41a6.el9 120 kB/s | 27 kB 00:00 2024-01-19T04:53:03.425 INFO:teuthology.orchestra.run.smithi031.stdout:(14/155): libcephfs-devel-19.0.0-766.g0edf41a6. 466 kB/s | 31 kB 00:00 2024-01-19T04:53:03.425 INFO:teuthology.orchestra.run.smithi137.stdout:(27/155): rbd-nbd-19.0.0-766.g0edf41a6.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-01-19T04:53:03.517 INFO:teuthology.orchestra.run.smithi137.stdout:(28/155): ceph-grafana-dashboards-19.0.0-766.g0 310 kB/s | 28 kB 00:00 2024-01-19T04:53:03.518 INFO:teuthology.orchestra.run.smithi031.stdout:(15/155): libcephsqlite-19.0.0-766.g0edf41a6.el 1.7 MB/s | 164 kB 00:00 2024-01-19T04:53:03.551 INFO:teuthology.orchestra.run.smithi031.stdout:(16/155): libcephfs2-19.0.0-766.g0edf41a6.el9.x 4.3 MB/s | 742 kB 00:00 2024-01-19T04:53:03.592 INFO:teuthology.orchestra.run.smithi137.stdout:(29/155): ceph-mgr-cephadm-19.0.0-766.g0edf41a6 1.7 MB/s | 133 kB 00:00 2024-01-19T04:53:03.601 INFO:teuthology.orchestra.run.smithi031.stdout:(17/155): librados-devel-19.0.0-766.g0edf41a6.e 1.5 MB/s | 127 kB 00:00 2024-01-19T04:53:03.651 INFO:teuthology.orchestra.run.smithi031.stdout:(18/155): libradosstriper1-19.0.0-766.g0edf41a6 4.9 MB/s | 499 kB 00:00 2024-01-19T04:53:03.743 INFO:teuthology.orchestra.run.smithi031.stdout:(19/155): python3-ceph-argparse-19.0.0-766.g0ed 493 kB/s | 45 kB 00:00 2024-01-19T04:53:03.826 INFO:teuthology.orchestra.run.smithi137.stdout:(30/155): rbd-mirror-19.0.0-766.g0edf41a6.el9.x 5.2 MB/s | 3.1 MB 00:00 2024-01-19T04:53:03.835 INFO:teuthology.orchestra.run.smithi031.stdout:(20/155): python3-ceph-common-19.0.0-766.g0edf4 1.4 MB/s | 130 kB 00:00 2024-01-19T04:53:03.927 INFO:teuthology.orchestra.run.smithi031.stdout:(21/155): python3-cephfs-19.0.0-766.g0edf41a6.e 1.7 MB/s | 161 kB 00:00 2024-01-19T04:53:04.036 INFO:teuthology.orchestra.run.smithi031.stdout:(22/155): python3-rados-19.0.0-766.g0edf41a6.el 2.9 MB/s | 321 kB 00:00 2024-01-19T04:53:04.136 INFO:teuthology.orchestra.run.smithi031.stdout:(23/155): python3-rbd-19.0.0-766.g0edf41a6.el9. 2.9 MB/s | 298 kB 00:00 2024-01-19T04:53:04.218 INFO:teuthology.orchestra.run.smithi137.stdout:(31/155): ceph-mgr-dashboard-19.0.0-766.g0edf41 2.8 MB/s | 1.7 MB 00:00 2024-01-19T04:53:04.228 INFO:teuthology.orchestra.run.smithi031.stdout:(24/155): python3-rgw-19.0.0-766.g0edf41a6.el9. 1.1 MB/s | 99 kB 00:00 2024-01-19T04:53:04.320 INFO:teuthology.orchestra.run.smithi031.stdout:(25/155): rbd-fuse-19.0.0-766.g0edf41a6.el9.x86 931 kB/s | 85 kB 00:00 2024-01-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi137.stdout:(32/155): ceph-mgr-modules-core-19.0.0-766.g0ed 1.7 MB/s | 245 kB 00:00 2024-01-19T04:53:04.460 INFO:teuthology.orchestra.run.smithi137.stdout:(33/155): ceph-mgr-rook-19.0.0-766.g0edf41a6.el 507 kB/s | 51 kB 00:00 2024-01-19T04:53:04.560 INFO:teuthology.orchestra.run.smithi137.stdout:(34/155): ceph-prometheus-alerts-19.0.0-766.g0e 151 kB/s | 15 kB 00:00 2024-01-19T04:53:04.887 INFO:teuthology.orchestra.run.smithi031.stdout:(26/155): rbd-mirror-19.0.0-766.g0edf41a6.el9.x 5.4 MB/s | 3.1 MB 00:00 2024-01-19T04:53:04.911 INFO:teuthology.orchestra.run.smithi137.stdout:(35/155): ceph-volume-19.0.0-766.g0edf41a6.el9. 750 kB/s | 262 kB 00:00 2024-01-19T04:53:04.987 INFO:teuthology.orchestra.run.smithi031.stdout:(27/155): librgw2-19.0.0-766.g0edf41a6.el9.x86_ 3.5 MB/s | 4.8 MB 00:01 2024-01-19T04:53:05.012 INFO:teuthology.orchestra.run.smithi031.stdout:(28/155): rbd-nbd-19.0.0-766.g0edf41a6.el9.x86_ 1.3 MB/s | 170 kB 00:00 2024-01-19T04:53:05.046 INFO:teuthology.orchestra.run.smithi031.stdout:(29/155): ceph-grafana-dashboards-19.0.0-766.g0 485 kB/s | 28 kB 00:00 2024-01-19T04:53:05.080 INFO:teuthology.orchestra.run.smithi031.stdout:(30/155): ceph-mgr-cephadm-19.0.0-766.g0edf41a6 1.9 MB/s | 133 kB 00:00 2024-01-19T04:53:05.136 INFO:teuthology.orchestra.run.smithi137.stdout:(36/155): cephadm-19.0.0-766.g0edf41a6.el9.noar 3.2 MB/s | 742 kB 00:00 2024-01-19T04:53:05.211 INFO:teuthology.orchestra.run.smithi137.stdout:(37/155): cephfs-top-19.0.0-766.g0edf41a6.el9.n 244 kB/s | 18 kB 00:00 2024-01-19T04:53:05.370 INFO:teuthology.orchestra.run.smithi137.stdout:(38/155): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-01-19T04:53:05.513 INFO:teuthology.orchestra.run.smithi031.stdout:(31/155): ceph-mgr-dashboard-19.0.0-766.g0edf41 3.7 MB/s | 1.7 MB 00:00 2024-01-19T04:53:05.520 INFO:teuthology.orchestra.run.smithi137.stdout:(39/155): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 662 kB 00:00 2024-01-19T04:53:05.570 INFO:teuthology.orchestra.run.smithi137.stdout:(40/155): bison-3.7.4-5.el9.x86_64.rpm 4.5 MB/s | 920 kB 00:00 2024-01-19T04:53:05.595 INFO:teuthology.orchestra.run.smithi137.stdout:(41/155): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-01-19T04:53:05.612 INFO:teuthology.orchestra.run.smithi137.stdout:(42/155): elfutils-libelf-devel-0.190-2.el9.x86 490 kB/s | 20 kB 00:00 2024-01-19T04:53:05.638 INFO:teuthology.orchestra.run.smithi031.stdout:(32/155): ceph-mgr-modules-core-19.0.0-766.g0ed 1.9 MB/s | 245 kB 00:00 2024-01-19T04:53:05.646 INFO:teuthology.orchestra.run.smithi137.stdout:(43/155): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-01-19T04:53:05.663 INFO:teuthology.orchestra.run.smithi137.stdout:(44/155): flexiblas-3.0.4-8.el9.x86_64.rpm 629 kB/s | 32 kB 00:00 2024-01-19T04:53:05.697 INFO:teuthology.orchestra.run.smithi031.stdout:(33/155): ceph-mgr-rook-19.0.0-766.g0edf41a6.el 876 kB/s | 51 kB 00:00 2024-01-19T04:53:05.739 INFO:teuthology.orchestra.run.smithi031.stdout:(34/155): ceph-prometheus-alerts-19.0.0-766.g0e 362 kB/s | 15 kB 00:00 2024-01-19T04:53:05.763 INFO:teuthology.orchestra.run.smithi137.stdout:(45/155): flexiblas-openblas-openmp-3.0.4-8.el9 171 kB/s | 17 kB 00:00 2024-01-19T04:53:05.856 INFO:teuthology.orchestra.run.smithi031.stdout:(35/155): ceph-volume-19.0.0-766.g0edf41a6.el9. 2.2 MB/s | 262 kB 00:00 2024-01-19T04:53:05.864 INFO:teuthology.orchestra.run.smithi137.stdout:(46/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-01-19T04:53:05.889 INFO:teuthology.orchestra.run.smithi137.stdout:(47/155): libaio-devel-0.3.111-13.el9.x86_64.rp 81 kB/s | 10 kB 00:00 2024-01-19T04:53:05.922 INFO:teuthology.orchestra.run.smithi137.stdout:(48/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-01-19T04:53:05.948 INFO:teuthology.orchestra.run.smithi137.stdout:(49/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-01-19T04:53:05.998 INFO:teuthology.orchestra.run.smithi137.stdout:(50/155): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-01-19T04:53:06.031 INFO:teuthology.orchestra.run.smithi137.stdout:(51/155): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-01-19T04:53:06.065 INFO:teuthology.orchestra.run.smithi137.stdout:(52/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 369 kB/s | 24 kB 00:00 2024-01-19T04:53:06.091 INFO:teuthology.orchestra.run.smithi137.stdout:(53/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 787 kB/s | 46 kB 00:00 2024-01-19T04:53:06.116 INFO:teuthology.orchestra.run.smithi137.stdout:(54/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-01-19T04:53:06.141 INFO:teuthology.orchestra.run.smithi137.stdout:(55/155): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-01-19T04:53:06.166 INFO:teuthology.orchestra.run.smithi137.stdout:(56/155): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-01-19T04:53:06.365 INFO:teuthology.orchestra.run.smithi031.stdout:(36/155): ceph-test-19.0.0-766.g0edf41a6.el9.x8 10 MB/s | 46 MB 00:04 2024-01-19T04:53:06.383 INFO:teuthology.orchestra.run.smithi137.stdout:(57/155): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-01-19T04:53:06.406 INFO:teuthology.orchestra.run.smithi031.stdout:(37/155): cephadm-19.0.0-766.g0edf41a6.el9.noar 1.3 MB/s | 742 kB 00:00 2024-01-19T04:53:06.432 INFO:teuthology.orchestra.run.smithi031.stdout:(38/155): cephfs-top-19.0.0-766.g0edf41a6.el9.n 274 kB/s | 18 kB 00:00 2024-01-19T04:53:06.442 INFO:teuthology.orchestra.run.smithi137.stdout:(58/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-01-19T04:53:06.485 INFO:teuthology.orchestra.run.smithi137.stdout:(59/155): postgresql-13.11-1.el9.x86_64.rpm 16 MB/s | 1.5 MB 00:00 2024-01-19T04:53:06.515 INFO:teuthology.orchestra.run.smithi031.stdout:(39/155): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-01-19T04:53:06.594 INFO:teuthology.orchestra.run.smithi137.stdout:(60/155): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-01-19T04:53:06.594 INFO:teuthology.orchestra.run.smithi137.stdout:(61/155): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-01-19T04:53:06.709 INFO:teuthology.orchestra.run.smithi031.stdout:(40/155): boost-program-options-1.75.0-8.el9.x8 771 kB/s | 106 kB 00:00 2024-01-19T04:53:06.727 INFO:teuthology.orchestra.run.smithi137.stdout:(62/155): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-01-19T04:53:06.752 INFO:teuthology.orchestra.run.smithi137.stdout:(63/155): python3-attrs-20.3.0-7.el9.noarch.rpm 373 kB/s | 78 kB 00:00 2024-01-19T04:53:06.758 INFO:teuthology.orchestra.run.smithi031.stdout:(41/155): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 920 kB 00:00 2024-01-19T04:53:06.799 INFO:teuthology.orchestra.run.smithi031.stdout:(42/155): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-01-19T04:53:06.831 INFO:teuthology.orchestra.run.smithi031.stdout:(43/155): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 20 kB 00:00 2024-01-19T04:53:06.977 INFO:teuthology.orchestra.run.smithi031.stdout:(44/155): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 307 kB 00:00 2024-01-19T04:53:06.994 INFO:teuthology.orchestra.run.smithi031.stdout:(45/155): flexiblas-3.0.4-8.el9.x86_64.rpm 163 kB/s | 32 kB 00:00 2024-01-19T04:53:07.027 INFO:teuthology.orchestra.run.smithi137.stdout:(64/155): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-01-19T04:53:07.053 INFO:teuthology.orchestra.run.smithi137.stdout:(65/155): python3-jinja2-2.11.3-4.el9.noarch.rp 762 kB/s | 228 kB 00:00 2024-01-19T04:53:07.094 INFO:teuthology.orchestra.run.smithi137.stdout:(66/155): python3-jsonpointer-2.0-4.el9.noarch. 436 kB/s | 18 kB 00:00 2024-01-19T04:53:07.119 INFO:teuthology.orchestra.run.smithi137.stdout:(67/155): python3-jsonpatch-1.21-16.el9.noarch. 366 kB/s | 24 kB 00:00 2024-01-19T04:53:07.145 INFO:teuthology.orchestra.run.smithi137.stdout:(68/155): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-01-19T04:53:07.170 INFO:teuthology.orchestra.run.smithi137.stdout:(69/155): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-01-19T04:53:07.219 INFO:teuthology.orchestra.run.smithi031.stdout:(46/155): flexiblas-openblas-openmp-3.0.4-8.el9 76 kB/s | 17 kB 00:00 2024-01-19T04:53:07.236 INFO:teuthology.orchestra.run.smithi031.stdout:(47/155): libaio-devel-0.3.111-13.el9.x86_64.rp 42 kB/s | 10 kB 00:00 2024-01-19T04:53:07.319 INFO:teuthology.orchestra.run.smithi031.stdout:(48/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-01-19T04:53:07.329 INFO:teuthology.orchestra.run.smithi137.stdout:(70/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-01-19T04:53:07.345 INFO:teuthology.orchestra.run.smithi031.stdout:(49/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-01-19T04:53:07.354 INFO:teuthology.orchestra.run.smithi137.stdout:(71/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-01-19T04:53:07.370 INFO:teuthology.orchestra.run.smithi031.stdout:(50/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 333 kB/s | 44 kB 00:00 2024-01-19T04:53:07.379 INFO:teuthology.orchestra.run.smithi137.stdout:(72/155): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-01-19T04:53:07.404 INFO:teuthology.orchestra.run.smithi031.stdout:(51/155): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-01-19T04:53:07.404 INFO:teuthology.orchestra.run.smithi137.stdout:(73/155): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-01-19T04:53:07.430 INFO:teuthology.orchestra.run.smithi137.stdout:(74/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-01-19T04:53:07.463 INFO:teuthology.orchestra.run.smithi031.stdout:(52/155): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-01-19T04:53:07.463 INFO:teuthology.orchestra.run.smithi137.stdout:(75/155): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2024-01-19T04:53:07.487 INFO:teuthology.orchestra.run.smithi031.stdout:(53/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 211 kB/s | 24 kB 00:00 2024-01-19T04:53:07.488 INFO:teuthology.orchestra.run.smithi137.stdout:(76/155): python3-pytz-2021.1-5.el9.noarch.rpm 807 kB/s | 47 kB 00:00 2024-01-19T04:53:07.513 INFO:teuthology.orchestra.run.smithi031.stdout:(54/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 427 kB/s | 46 kB 00:00 2024-01-19T04:53:07.514 INFO:teuthology.orchestra.run.smithi137.stdout:(77/155): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-01-19T04:53:07.553 INFO:teuthology.orchestra.run.smithi031.stdout:(55/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 282 kB 00:00 2024-01-19T04:53:07.571 INFO:teuthology.orchestra.run.smithi031.stdout:(56/155): openblas-0.3.21-2.el9.x86_64.rpm 591 kB/s | 34 kB 00:00 2024-01-19T04:53:07.605 INFO:teuthology.orchestra.run.smithi031.stdout:(57/155): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-01-19T04:53:07.756 INFO:teuthology.orchestra.run.smithi137.stdout:(78/155): python3-toml-0.10.2-6.el9.noarch.rpm 160 kB/s | 39 kB 00:00 2024-01-19T04:53:08.073 INFO:teuthology.orchestra.run.smithi137.stdout:(79/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-01-19T04:53:08.089 INFO:teuthology.orchestra.run.smithi137.stdout:(80/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 123 kB/s | 41 kB 00:00 2024-01-19T04:53:08.122 INFO:teuthology.orchestra.run.smithi031.stdout:(58/155): openblas-openmp-0.3.21-2.el9.x86_64.r 8.3 MB/s | 4.7 MB 00:00 2024-01-19T04:53:08.123 INFO:teuthology.orchestra.run.smithi137.stdout:(81/155): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-01-19T04:53:08.165 INFO:teuthology.orchestra.run.smithi137.stdout:(82/155): uuid-1.6.2-55.el9.x86_64.rpm 749 kB/s | 56 kB 00:00 2024-01-19T04:53:08.172 INFO:teuthology.orchestra.run.smithi031.stdout:(59/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 5.0 MB/s | 3.0 MB 00:00 2024-01-19T04:53:08.191 INFO:teuthology.orchestra.run.smithi137.stdout:(83/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2024-01-19T04:53:08.214 INFO:teuthology.orchestra.run.smithi031.stdout:(60/155): postgresql-13.11-1.el9.x86_64.rpm 2.5 MB/s | 1.5 MB 00:00 2024-01-19T04:53:08.216 INFO:teuthology.orchestra.run.smithi137.stdout:(84/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-01-19T04:53:08.239 INFO:teuthology.orchestra.run.smithi031.stdout:(61/155): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-01-19T04:53:08.241 INFO:teuthology.orchestra.run.smithi137.stdout:(85/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-01-19T04:53:08.266 INFO:teuthology.orchestra.run.smithi137.stdout:(86/155): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-01-19T04:53:08.273 INFO:teuthology.orchestra.run.smithi031.stdout:(62/155): postgresql-contrib-13.11-1.el9.x86_64 7.9 MB/s | 811 kB 00:00 2024-01-19T04:53:08.367 INFO:teuthology.orchestra.run.smithi137.stdout:(87/155): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 802 kB 00:00 2024-01-19T04:53:08.409 INFO:teuthology.orchestra.run.smithi137.stdout:(88/155): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 186 kB 00:00 2024-01-19T04:53:08.443 INFO:teuthology.orchestra.run.smithi137.stdout:(89/155): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-01-19T04:53:08.457 INFO:teuthology.orchestra.run.smithi031.stdout:(63/155): python3-attrs-20.3.0-7.el9.noarch.rpm 359 kB/s | 78 kB 00:00 2024-01-19T04:53:08.468 INFO:teuthology.orchestra.run.smithi137.stdout:(90/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-01-19T04:53:08.501 INFO:teuthology.orchestra.run.smithi137.stdout:(91/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-01-19T04:53:08.524 INFO:teuthology.orchestra.run.smithi031.stdout:(64/155): postgresql-server-13.11-1.el9.x86_64. 18 MB/s | 5.7 MB 00:00 2024-01-19T04:53:08.560 INFO:teuthology.orchestra.run.smithi137.stdout:(92/155): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-01-19T04:53:08.585 INFO:teuthology.orchestra.run.smithi137.stdout:(93/155): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-01-19T04:53:08.611 INFO:teuthology.orchestra.run.smithi137.stdout:(94/155): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-01-19T04:53:08.636 INFO:teuthology.orchestra.run.smithi137.stdout:(95/155): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2024-01-19T04:53:08.661 INFO:teuthology.orchestra.run.smithi137.stdout:(96/155): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-01-19T04:53:08.682 INFO:teuthology.orchestra.run.smithi031.stdout:(65/155): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2024-01-19T04:53:08.716 INFO:teuthology.orchestra.run.smithi031.stdout:(66/155): python3-jinja2-2.11.3-4.el9.noarch.rp 889 kB/s | 228 kB 00:00 2024-01-19T04:53:08.741 INFO:teuthology.orchestra.run.smithi031.stdout:(67/155): python3-jsonpatch-1.21-16.el9.noarch. 114 kB/s | 24 kB 00:00 2024-01-19T04:53:08.766 INFO:teuthology.orchestra.run.smithi031.stdout:(68/155): python3-jsonpointer-2.0-4.el9.noarch. 217 kB/s | 18 kB 00:00 2024-01-19T04:53:08.786 INFO:teuthology.orchestra.run.smithi137.stdout:(97/155): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-01-19T04:53:08.791 INFO:teuthology.orchestra.run.smithi031.stdout:(69/155): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2024-01-19T04:53:08.817 INFO:teuthology.orchestra.run.smithi031.stdout:(70/155): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-01-19T04:53:08.937 INFO:teuthology.orchestra.run.smithi137.stdout:(98/155): python3-devel-3.9.17-2.el9.x86_64.rpm 817 kB/s | 245 kB 00:00 2024-01-19T04:53:08.992 INFO:teuthology.orchestra.run.smithi031.stdout:(71/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-01-19T04:53:09.018 INFO:teuthology.orchestra.run.smithi031.stdout:(72/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 966 kB/s | 217 kB 00:00 2024-01-19T04:53:09.043 INFO:teuthology.orchestra.run.smithi031.stdout:(73/155): python3-oauthlib-3.1.1-5.el9.noarch.r 778 kB/s | 174 kB 00:00 2024-01-19T04:53:09.068 INFO:teuthology.orchestra.run.smithi031.stdout:(74/155): python3-prettytable-0.7.2-27.el9.noar 550 kB/s | 41 kB 00:00 2024-01-19T04:53:09.082 INFO:teuthology.orchestra.run.smithi137.stdout:(99/155): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.0 MB/s | 309 kB 00:00 2024-01-19T04:53:09.129 INFO:teuthology.orchestra.run.smithi137.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 565 kB/s | 26 kB 00:00 2024-01-19T04:53:09.171 INFO:teuthology.orchestra.run.smithi137.stdout:(101/155): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-01-19T04:53:09.202 INFO:teuthology.orchestra.run.smithi031.stdout:(75/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 833 kB/s | 132 kB 00:00 2024-01-19T04:53:09.213 INFO:teuthology.orchestra.run.smithi137.stdout:(102/155): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-01-19T04:53:09.227 INFO:teuthology.orchestra.run.smithi031.stdout:(76/155): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 211 kB 00:00 2024-01-19T04:53:09.253 INFO:teuthology.orchestra.run.smithi031.stdout:(77/155): python3-pytz-2021.1-5.el9.noarch.rpm 257 kB/s | 47 kB 00:00 2024-01-19T04:53:09.271 INFO:teuthology.orchestra.run.smithi137.stdout:(103/155): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-01-19T04:53:09.278 INFO:teuthology.orchestra.run.smithi031.stdout:(78/155): python3-requests-oauthlib-1.3.0-12.el 858 kB/s | 43 kB 00:00 2024-01-19T04:53:09.478 INFO:teuthology.orchestra.run.smithi031.stdout:(79/155): python3-toml-0.10.2-6.el9.noarch.rpm 172 kB/s | 39 kB 00:00 2024-01-19T04:53:09.503 INFO:teuthology.orchestra.run.smithi031.stdout:(80/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 183 kB/s | 41 kB 00:00 2024-01-19T04:53:09.597 INFO:teuthology.orchestra.run.smithi137.stdout:(104/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-01-19T04:53:09.754 INFO:teuthology.orchestra.run.smithi031.stdout:(81/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-01-19T04:53:09.755 INFO:teuthology.orchestra.run.smithi137.stdout:(105/155): python3-asyncssh-2.13.2-3.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-01-19T04:53:09.779 INFO:teuthology.orchestra.run.smithi031.stdout:(82/155): socat-1.7.4.1-5.el9.x86_64.rpm 998 kB/s | 300 kB 00:00 2024-01-19T04:53:09.804 INFO:teuthology.orchestra.run.smithi031.stdout:(83/155): uuid-1.6.2-55.el9.x86_64.rpm 188 kB/s | 56 kB 00:00 2024-01-19T04:53:09.805 INFO:teuthology.orchestra.run.smithi137.stdout:(106/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 873 kB/s | 43 kB 00:00 2024-01-19T04:53:09.829 INFO:teuthology.orchestra.run.smithi031.stdout:(84/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2024-01-19T04:53:09.855 INFO:teuthology.orchestra.run.smithi031.stdout:(85/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-01-19T04:53:09.855 INFO:teuthology.orchestra.run.smithi137.stdout:(107/155): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-01-19T04:53:09.880 INFO:teuthology.orchestra.run.smithi031.stdout:(86/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-01-19T04:53:09.905 INFO:teuthology.orchestra.run.smithi031.stdout:(87/155): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-01-19T04:53:09.906 INFO:teuthology.orchestra.run.smithi137.stdout:(108/155): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-01-19T04:53:09.930 INFO:teuthology.orchestra.run.smithi031.stdout:(88/155): jq-1.6-16.el9.x86_64.rpm 3.7 MB/s | 186 kB 00:00 2024-01-19T04:53:09.975 INFO:teuthology.orchestra.run.smithi031.stdout:(89/155): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 802 kB 00:00 2024-01-19T04:53:09.990 INFO:teuthology.orchestra.run.smithi137.stdout:(109/155): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-01-19T04:53:09.997 INFO:teuthology.orchestra.run.smithi031.stdout:(90/155): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 194 kB 00:00 2024-01-19T04:53:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:(91/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2024-01-19T04:53:10.048 INFO:teuthology.orchestra.run.smithi031.stdout:(92/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-01-19T04:53:10.082 INFO:teuthology.orchestra.run.smithi031.stdout:(93/155): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-01-19T04:53:10.107 INFO:teuthology.orchestra.run.smithi031.stdout:(94/155): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-01-19T04:53:10.115 INFO:teuthology.orchestra.run.smithi137.stdout:(110/155): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-01-19T04:53:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:(95/155): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-01-19T04:53:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:(96/155): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2024-01-19T04:53:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:(97/155): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-01-19T04:53:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:(98/155): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-01-19T04:53:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:(99/155): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-01-19T04:53:10.584 INFO:teuthology.orchestra.run.smithi031.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-01-19T04:53:10.591 INFO:teuthology.orchestra.run.smithi137.stdout:(111/155): ceph-test-19.0.0-766.g0edf41a6.el9.x 4.3 MB/s | 46 MB 00:10 2024-01-19T04:53:10.616 INFO:teuthology.orchestra.run.smithi137.stdout:(112/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 606 kB/s | 303 kB 00:00 2024-01-19T04:53:10.641 INFO:teuthology.orchestra.run.smithi137.stdout:(113/155): python3-colorama-0.4.4-12.el9.noarch 706 kB/s | 35 kB 00:00 2024-01-19T04:53:10.700 INFO:teuthology.orchestra.run.smithi137.stdout:(114/155): python3-jaraco-8.2.1-3.el9.noarch.rp 333 kB/s | 11 kB 00:00 2024-01-19T04:53:10.734 INFO:teuthology.orchestra.run.smithi031.stdout:(101/155): gperftools-libs-2.9.1-2.el9.x86_64.r 561 kB/s | 309 kB 00:00 2024-01-19T04:53:10.759 INFO:teuthology.orchestra.run.smithi137.stdout:(115/155): libarrow-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-01-19T04:53:10.775 INFO:teuthology.orchestra.run.smithi137.stdout:(116/155): python3-jaraco-classes-3.2.1-5.el9.n 240 kB/s | 18 kB 00:00 2024-01-19T04:53:10.792 INFO:teuthology.orchestra.run.smithi137.stdout:(117/155): python3-jaraco-collections-3.0.0-8.e 696 kB/s | 23 kB 00:00 2024-01-19T04:53:10.792 INFO:teuthology.orchestra.run.smithi031.stdout:(102/155): liboath-2.6.7-2.el9.x86_64.rpm 233 kB/s | 49 kB 00:00 2024-01-19T04:53:10.809 INFO:teuthology.orchestra.run.smithi137.stdout:(118/155): python3-jaraco-functools-3.5.0-2.el9 580 kB/s | 19 kB 00:00 2024-01-19T04:53:10.826 INFO:teuthology.orchestra.run.smithi137.stdout:(119/155): python3-jaraco-text-3.2.0-6.el9.noar 584 kB/s | 20 kB 00:00 2024-01-19T04:53:10.843 INFO:teuthology.orchestra.run.smithi031.stdout:(103/155): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-01-19T04:53:10.868 INFO:teuthology.orchestra.run.smithi137.stdout:(120/155): python3-google-auth-2.26.0-1.el9.noa 802 kB/s | 202 kB 00:00 2024-01-19T04:53:10.893 INFO:teuthology.orchestra.run.smithi137.stdout:(121/155): python3-logutils-0.3.5-21.el9.noarch 696 kB/s | 46 kB 00:00 2024-01-19T04:53:10.919 INFO:teuthology.orchestra.run.smithi137.stdout:(122/155): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-01-19T04:53:10.944 INFO:teuthology.orchestra.run.smithi137.stdout:(123/155): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-01-19T04:53:10.986 INFO:teuthology.orchestra.run.smithi137.stdout:(124/155): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-01-19T04:53:11.011 INFO:teuthology.orchestra.run.smithi137.stdout:(125/155): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-01-19T04:53:11.043 INFO:teuthology.orchestra.run.smithi031.stdout:(104/155): luarocks-3.9.2-1.el9.noarch.rpm 603 kB/s | 151 kB 00:00 2024-01-19T04:53:11.053 INFO:teuthology.orchestra.run.smithi137.stdout:(126/155): python3-pyperclip-1.8.2-2.el9.noarch 683 kB/s | 28 kB 00:00 2024-01-19T04:53:11.070 INFO:teuthology.orchestra.run.smithi137.stdout:(127/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-01-19T04:53:11.094 INFO:teuthology.orchestra.run.smithi031.stdout:(105/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-01-19T04:53:11.095 INFO:teuthology.orchestra.run.smithi137.stdout:(128/155): python3-repoze-lru-0.7-16.el9.noarch 733 kB/s | 31 kB 00:00 2024-01-19T04:53:11.137 INFO:teuthology.orchestra.run.smithi137.stdout:(129/155): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-01-19T04:53:11.179 INFO:teuthology.orchestra.run.smithi137.stdout:(130/155): python3-tempora-5.0.0-2.el9.noarch.r 863 kB/s | 36 kB 00:00 2024-01-19T04:53:11.194 INFO:teuthology.orchestra.run.smithi031.stdout:(106/155): libarrow-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-01-19T04:53:11.204 INFO:teuthology.orchestra.run.smithi137.stdout:(131/155): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-01-19T04:53:11.219 INFO:teuthology.orchestra.run.smithi031.stdout:(107/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-01-19T04:53:11.238 INFO:teuthology.orchestra.run.smithi137.stdout:(132/155): python3-kubernetes-26.1.0-2.el9.noar 2.4 MB/s | 1.0 MB 00:00 2024-01-19T04:53:11.263 INFO:teuthology.orchestra.run.smithi137.stdout:(133/155): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-01-19T04:53:11.286 INFO:teuthology.orchestra.run.smithi031.stdout:(108/155): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-01-19T04:53:11.305 INFO:teuthology.orchestra.run.smithi137.stdout:(134/155): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-01-19T04:53:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:(109/155): python3-asyncssh-2.13.2-3.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-01-19T04:53:11.339 INFO:teuthology.orchestra.run.smithi137.stdout:(135/155): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-01-19T04:53:11.353 INFO:teuthology.orchestra.run.smithi031.stdout:(110/155): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-01-19T04:53:11.364 INFO:teuthology.orchestra.run.smithi137.stdout:(136/155): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-01-19T04:53:11.387 INFO:teuthology.orchestra.run.smithi031.stdout:(111/155): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-01-19T04:53:11.446 INFO:teuthology.orchestra.run.smithi031.stdout:(112/155): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-01-19T04:53:11.448 INFO:teuthology.orchestra.run.smithi137.stdout:(137/155): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-01-19T04:53:11.473 INFO:teuthology.orchestra.run.smithi137.stdout:(138/155): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-01-19T04:53:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:(113/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-01-19T04:53:11.505 INFO:teuthology.orchestra.run.smithi031.stdout:(114/155): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-01-19T04:53:11.555 INFO:teuthology.orchestra.run.smithi031.stdout:(115/155): python3-google-auth-2.26.0-1.el9.noa 1.8 MB/s | 202 kB 00:00 2024-01-19T04:53:11.580 INFO:teuthology.orchestra.run.smithi031.stdout:(116/155): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-01-19T04:53:11.605 INFO:teuthology.orchestra.run.smithi031.stdout:(117/155): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-01-19T04:53:11.656 INFO:teuthology.orchestra.run.smithi031.stdout:(118/155): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-01-19T04:53:11.681 INFO:teuthology.orchestra.run.smithi031.stdout:(119/155): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-01-19T04:53:11.706 INFO:teuthology.orchestra.run.smithi031.stdout:(120/155): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-01-19T04:53:11.781 INFO:teuthology.orchestra.run.smithi031.stdout:(121/155): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-01-19T04:53:11.807 INFO:teuthology.orchestra.run.smithi031.stdout:(122/155): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-01-19T04:53:11.840 INFO:teuthology.orchestra.run.smithi031.stdout:(123/155): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-01-19T04:53:11.891 INFO:teuthology.orchestra.run.smithi031.stdout:(124/155): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-01-19T04:53:11.907 INFO:teuthology.orchestra.run.smithi137.stdout:(139/155): librados2-19.0.0-766.g0edf41a6.el9.x 7.3 MB/s | 3.4 MB 00:00 2024-01-19T04:53:11.916 INFO:teuthology.orchestra.run.smithi031.stdout:(125/155): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-01-19T04:53:11.941 INFO:teuthology.orchestra.run.smithi031.stdout:(126/155): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-01-19T04:53:11.948 INFO:teuthology.orchestra.run.smithi137.stdout:(140/155): thrift-0.15.0-2.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-01-19T04:53:11.974 INFO:teuthology.orchestra.run.smithi137.stdout:(141/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 68 kB 00:00 2024-01-19T04:53:12.000 INFO:teuthology.orchestra.run.smithi031.stdout:(127/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-01-19T04:53:12.025 INFO:teuthology.orchestra.run.smithi031.stdout:(128/155): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-01-19T04:53:12.049 INFO:teuthology.orchestra.run.smithi137.stdout:(142/155): librbd1-19.0.0-766.g0edf41a6.el9.x86 5.4 MB/s | 3.1 MB 00:00 2024-01-19T04:53:12.050 INFO:teuthology.orchestra.run.smithi031.stdout:(129/155): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-01-19T04:53:12.108 INFO:teuthology.orchestra.run.smithi137.stdout:(143/155): NetworkManager-1.45.10-1.el9.x86_64. 14 MB/s | 2.2 MB 00:00 2024-01-19T04:53:12.142 INFO:teuthology.orchestra.run.smithi031.stdout:(130/155): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-01-19T04:53:12.183 INFO:teuthology.orchestra.run.smithi137.stdout:(144/155): NetworkManager-libnm-1.45.10-1.el9.x 8.6 MB/s | 1.8 MB 00:00 2024-01-19T04:53:12.208 INFO:teuthology.orchestra.run.smithi137.stdout:(145/155): NetworkManager-team-1.45.10-1.el9.x8 269 kB/s | 42 kB 00:00 2024-01-19T04:53:12.234 INFO:teuthology.orchestra.run.smithi137.stdout:(146/155): NetworkManager-tui-1.45.10-1.el9.x86 1.9 MB/s | 246 kB 00:00 2024-01-19T04:53:12.259 INFO:teuthology.orchestra.run.smithi137.stdout:(147/155): acl-2.3.1-4.el9.x86_64.rpm 922 kB/s | 69 kB 00:00 2024-01-19T04:53:12.268 INFO:teuthology.orchestra.run.smithi031.stdout:(131/155): python3-rsa-4.9-2.el9.noarch.rpm 244 kB/s | 59 kB 00:00 2024-01-19T04:53:12.284 INFO:teuthology.orchestra.run.smithi137.stdout:(148/155): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-01-19T04:53:12.293 INFO:teuthology.orchestra.run.smithi031.stdout:(132/155): python3-tempora-5.0.0-2.el9.noarch.r 147 kB/s | 36 kB 00:00 2024-01-19T04:53:12.310 INFO:teuthology.orchestra.run.smithi137.stdout:(149/155): elfutils-debuginfod-client-0.190-2.e 470 kB/s | 35 kB 00:00 2024-01-19T04:53:12.319 INFO:teuthology.orchestra.run.smithi031.stdout:(133/155): python3-typing-extensions-4.4.0-2.el 289 kB/s | 51 kB 00:00 2024-01-19T04:53:12.334 INFO:teuthology.orchestra.run.smithi137.stdout:(150/155): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 254 kB 00:00 2024-01-19T04:53:12.360 INFO:teuthology.orchestra.run.smithi137.stdout:(151/155): libacl-2.3.1-4.el9.x86_64.rpm 294 kB/s | 22 kB 00:00 2024-01-19T04:53:12.377 INFO:teuthology.orchestra.run.smithi031.stdout:(134/155): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-01-19T04:53:12.402 INFO:teuthology.orchestra.run.smithi137.stdout:(152/155): openssl-3.0.7-25.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-01-19T04:53:12.402 INFO:teuthology.orchestra.run.smithi031.stdout:(135/155): python3-websocket-client-1.2.3-2.el9 823 kB/s | 90 kB 00:00 2024-01-19T04:53:12.435 INFO:teuthology.orchestra.run.smithi137.stdout:(153/155): xfsdump-3.1.12-4.el9.x86_64.rpm 4.3 MB/s | 321 kB 00:00 2024-01-19T04:53:12.469 INFO:teuthology.orchestra.run.smithi031.stdout:(136/155): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-01-19T04:53:12.495 INFO:teuthology.orchestra.run.smithi031.stdout:(137/155): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-01-19T04:53:12.519 INFO:teuthology.orchestra.run.smithi137.stdout:(154/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 14 MB/s | 2.1 MB 00:00 2024-01-19T04:53:12.528 INFO:teuthology.orchestra.run.smithi031.stdout:(138/155): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-01-19T04:53:12.561 INFO:teuthology.orchestra.run.smithi137.stdout:(155/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.7 MB/s | 1.1 MB 00:00 2024-01-19T04:53:12.568 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-19T04:53:12.568 INFO:teuthology.orchestra.run.smithi137.stdout:Total 14 MB/s | 209 MB 00:15 2024-01-19T04:53:12.598 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-19T04:53:12.745 INFO:teuthology.orchestra.run.smithi031.stdout:(139/155): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-01-19T04:53:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:(140/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-01-19T04:53:12.867 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-19T04:53:12.867 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-19T04:53:12.912 INFO:teuthology.orchestra.run.smithi031.stdout:(141/155): NetworkManager-1.45.10-1.el9.x86_64. 18 MB/s | 2.2 MB 00:00 2024-01-19T04:53:12.996 INFO:teuthology.orchestra.run.smithi031.stdout:(142/155): librbd1-19.0.0-766.g0edf41a6.el9.x86 6.7 MB/s | 3.1 MB 00:00 2024-01-19T04:53:13.038 INFO:teuthology.orchestra.run.smithi031.stdout:(143/155): NetworkManager-libnm-1.45.10-1.el9.x 14 MB/s | 1.8 MB 00:00 2024-01-19T04:53:13.063 INFO:teuthology.orchestra.run.smithi031.stdout:(144/155): NetworkManager-team-1.45.10-1.el9.x8 646 kB/s | 42 kB 00:00 2024-01-19T04:53:13.097 INFO:teuthology.orchestra.run.smithi031.stdout:(145/155): NetworkManager-tui-1.45.10-1.el9.x86 4.1 MB/s | 246 kB 00:00 2024-01-19T04:53:13.122 INFO:teuthology.orchestra.run.smithi031.stdout:(146/155): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2024-01-19T04:53:13.148 INFO:teuthology.orchestra.run.smithi031.stdout:(147/155): elfutils-debuginfod-client-0.190-2.e 705 kB/s | 35 kB 00:00 2024-01-19T04:53:13.173 INFO:teuthology.orchestra.run.smithi031.stdout:(148/155): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-01-19T04:53:13.198 INFO:teuthology.orchestra.run.smithi031.stdout:(149/155): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-01-19T04:53:13.224 INFO:teuthology.orchestra.run.smithi031.stdout:(150/155): libacl-2.3.1-4.el9.x86_64.rpm 440 kB/s | 22 kB 00:00 2024-01-19T04:53:13.282 INFO:teuthology.orchestra.run.smithi031.stdout:(151/155): librados2-19.0.0-766.g0edf41a6.el9.x 4.3 MB/s | 3.4 MB 00:00 2024-01-19T04:53:13.324 INFO:teuthology.orchestra.run.smithi031.stdout:(152/155): openssl-3.0.7-25.el9.x86_64.rpm 9.2 MB/s | 1.2 MB 00:00 2024-01-19T04:53:13.374 INFO:teuthology.orchestra.run.smithi031.stdout:(153/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 14 MB/s | 2.1 MB 00:00 2024-01-19T04:53:13.408 INFO:teuthology.orchestra.run.smithi031.stdout:(154/155): xfsdump-3.1.12-4.el9.x86_64.rpm 2.6 MB/s | 321 kB 00:00 2024-01-19T04:53:13.452 INFO:teuthology.orchestra.run.smithi031.stdout:(155/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-01-19T04:53:13.456 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-19T04:53:13.456 INFO:teuthology.orchestra.run.smithi031.stdout:Total 14 MB/s | 209 MB 00:14 2024-01-19T04:53:13.488 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-19T04:53:13.783 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-19T04:53:13.783 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-19T04:53:14.862 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-19T04:53:14.862 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-19T04:53:15.727 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-19T04:53:15.727 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-19T04:53:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-19T04:53:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-19T04:53:21.090 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-19T04:53:21.157 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-19T04:53:21.190 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-19T04:53:21.215 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-19T04:53:21.265 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-19T04:53:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-19T04:53:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 8/171 2024-01-19T04:53:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 8/171 2024-01-19T04:53:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 9/171 2024-01-19T04:53:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-19T04:53:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 9/171 2024-01-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-766.g0edf41a6.el9.x86_64 10/171 2024-01-19T04:53:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-19T04:53:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-19T04:53:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-19T04:53:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-19T04:53:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-19T04:53:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-19T04:53:22.383 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-19T04:53:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-19T04:53:22.439 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-19T04:53:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-19T04:53:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 15/171 2024-01-19T04:53:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 15/171 2024-01-19T04:53:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_ 16/171 2024-01-19T04:53:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_ 16/171 2024-01-19T04:53:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-19T04:53:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-19T04:53:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 8/171 2024-01-19T04:53:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-19T04:53:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 8/171 2024-01-19T04:53:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-19T04:53:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 9/171 2024-01-19T04:53:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 9/171 2024-01-19T04:53:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-19T04:53:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.0.0-766.g0edf41a6.el9.x86_64 10/171 2024-01-19T04:53:23.172 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-19T04:53:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-19T04:53:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-19T04:53:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-19T04:53:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-19T04:53:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-19T04:53:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-19T04:53:23.595 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-19T04:53:23.631 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 15/171 2024-01-19T04:53:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/171 2024-01-19T04:53:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/171 2024-01-19T04:53:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/171 2024-01-19T04:53:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 15/171 2024-01-19T04:53:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/171 2024-01-19T04:53:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_ 16/171 2024-01-19T04:53:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/171 2024-01-19T04:53:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_ 16/171 2024-01-19T04:53:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/171 2024-01-19T04:53:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/171 2024-01-19T04:53:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/171 2024-01-19T04:53:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/171 2024-01-19T04:53:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-19T04:53:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-766.g0edf41a6.el9.x 31/171 2024-01-19T04:53:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-766.g0edf41a6.el9 32/171 2024-01-19T04:53:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-766.g0edf41a6.el9.x86_64 33/171 2024-01-19T04:53:24.128 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-19T04:53:24.166 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-19T04:53:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-19T04:53:24.251 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-19T04:53:24.277 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-19T04:53:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-19T04:53:24.387 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-19T04:53:24.446 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-19T04:53:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-19T04:53:24.669 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-19T04:53:24.740 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-19T04:53:24.801 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/171 2024-01-19T04:53:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/171 2024-01-19T04:53:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/171 2024-01-19T04:53:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/171 2024-01-19T04:53:24.959 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-19T04:53:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/171 2024-01-19T04:53:25.023 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/171 2024-01-19T04:53:25.041 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-19T04:53:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/171 2024-01-19T04:53:25.080 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-19T04:53:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/171 2024-01-19T04:53:25.131 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-19T04:53:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/171 2024-01-19T04:53:25.168 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 42/171 2024-01-19T04:53:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.0.0-766.g0edf41a6.el9.x 31/171 2024-01-19T04:53:25.215 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-19T04:53:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.0.0-766.g0edf41a6.el9 32/171 2024-01-19T04:53:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.0.0-766.g0edf41a6.el9.x86_64 33/171 2024-01-19T04:53:25.333 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-19T04:53:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-19T04:53:25.395 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-19T04:53:25.419 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-19T04:53:25.501 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-19T04:53:25.539 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-19T04:53:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-19T04:53:25.593 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-19T04:53:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-19T04:53:25.666 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-19T04:53:25.698 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-19T04:53:25.726 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-19T04:53:25.772 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-19T04:53:25.808 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-19T04:53:25.852 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-19T04:53:25.893 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-19T04:53:26.026 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-19T04:53:26.074 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-19T04:53:26.104 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-19T04:53:26.148 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-19T04:53:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-19T04:53:26.239 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-19T04:53:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-19T04:53:26.279 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 61/171 2024-01-19T04:53:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-19T04:53:26.320 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-19T04:53:26.353 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-19T04:53:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-19T04:53:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 42/171 2024-01-19T04:53:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-19T04:53:26.446 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-19T04:53:26.489 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 65/171 2024-01-19T04:53:26.537 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 66/171 2024-01-19T04:53:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-19T04:53:26.592 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 67/171 2024-01-19T04:53:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-19T04:53:26.728 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-19T04:53:26.762 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-19T04:53:26.784 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 68/171 2024-01-19T04:53:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-19T04:53:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-19T04:53:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-19T04:53:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-19T04:53:27.003 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 69/171 2024-01-19T04:53:27.021 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-19T04:53:27.057 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-19T04:53:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-19T04:53:27.139 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 70/171 2024-01-19T04:53:27.143 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-19T04:53:27.175 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 71/171 2024-01-19T04:53:27.208 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/171 2024-01-19T04:53:27.246 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/171 2024-01-19T04:53:27.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-19T04:53:27.283 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-16.el9.x86_64 74/171 2024-01-19T04:53:27.314 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/171 2024-01-19T04:53:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-19T04:53:27.342 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/171 2024-01-19T04:53:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-19T04:53:27.380 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/171 2024-01-19T04:53:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-19T04:53:27.415 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/171 2024-01-19T04:53:27.447 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 79/171 2024-01-19T04:53:27.477 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-19T04:53:27.508 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-19T04:53:27.552 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 81/171 2024-01-19T04:53:27.587 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 82/171 2024-01-19T04:53:27.620 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 83/171 2024-01-19T04:53:27.653 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 84/171 2024-01-19T04:53:27.688 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 85/171 2024-01-19T04:53:27.721 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 61/171 2024-01-19T04:53:27.762 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-19T04:53:27.776 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 86/171 2024-01-19T04:53:27.794 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-19T04:53:27.807 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 87/171 2024-01-19T04:53:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-19T04:53:27.908 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 88/171 2024-01-19T04:53:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 65/171 2024-01-19T04:53:27.980 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 89/171 2024-01-19T04:53:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 66/171 2024-01-19T04:53:28.051 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 67/171 2024-01-19T04:53:28.254 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 68/171 2024-01-19T04:53:28.301 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 69/171 2024-01-19T04:53:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 70/171 2024-01-19T04:53:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 71/171 2024-01-19T04:53:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/171 2024-01-19T04:53:28.578 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/171 2024-01-19T04:53:28.616 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.6-16.el9.x86_64 74/171 2024-01-19T04:53:28.647 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/171 2024-01-19T04:53:28.675 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/171 2024-01-19T04:53:28.703 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-19T04:53:28.726 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/171 2024-01-19T04:53:28.750 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 91/171 2024-01-19T04:53:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/171 2024-01-19T04:53:28.784 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 92/171 2024-01-19T04:53:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 79/171 2024-01-19T04:53:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-19T04:53:28.832 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 93/171 2024-01-19T04:53:28.906 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 81/171 2024-01-19T04:53:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 94/171 2024-01-19T04:53:28.945 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 82/171 2024-01-19T04:53:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 95/171 2024-01-19T04:53:28.979 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 83/171 2024-01-19T04:53:29.023 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 84/171 2024-01-19T04:53:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 85/171 2024-01-19T04:53:29.137 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/171 2024-01-19T04:53:29.148 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 86/171 2024-01-19T04:53:29.166 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/171 2024-01-19T04:53:29.182 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 87/171 2024-01-19T04:53:29.245 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/171 2024-01-19T04:53:29.298 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 88/171 2024-01-19T04:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 89/171 2024-01-19T04:53:29.888 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/171 2024-01-19T04:53:29.995 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/171 2024-01-19T04:53:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-19T04:53:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 91/171 2024-01-19T04:53:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 92/171 2024-01-19T04:53:30.625 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 93/171 2024-01-19T04:53:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 94/171 2024-01-19T04:53:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 95/171 2024-01-19T04:53:30.933 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/171 2024-01-19T04:53:30.963 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/171 2024-01-19T04:53:31.043 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/171 2024-01-19T04:53:31.220 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/171 2024-01-19T04:53:31.287 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/171 2024-01-19T04:53:31.314 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 103/171 2024-01-19T04:53:31.348 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 104/171 2024-01-19T04:53:31.663 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/171 2024-01-19T04:53:31.694 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 106/171 2024-01-19T04:53:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/171 2024-01-19T04:53:31.801 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 106/171 2024-01-19T04:53:31.831 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-766.g0edf41a6.el9.x86_64 107/171 2024-01-19T04:53:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/171 2024-01-19T04:53:31.870 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/171 2024-01-19T04:53:32.294 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 109/171 2024-01-19T04:53:32.322 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 110/171 2024-01-19T04:53:32.393 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 110/171 2024-01-19T04:53:32.427 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-766.g0edf41a6.el9.x86_64 111/171 2024-01-19T04:53:33.172 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/171 2024-01-19T04:53:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/171 2024-01-19T04:53:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 103/171 2024-01-19T04:53:33.307 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 104/171 2024-01-19T04:53:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/171 2024-01-19T04:53:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 106/171 2024-01-19T04:53:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 106/171 2024-01-19T04:53:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.0.0-766.g0edf41a6.el9.x86_64 107/171 2024-01-19T04:53:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/171 2024-01-19T04:53:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 109/171 2024-01-19T04:53:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 110/171 2024-01-19T04:53:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 110/171 2024-01-19T04:53:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.0.0-766.g0edf41a6.el9.x86_64 111/171 2024-01-19T04:53:34.499 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 112/171 2024-01-19T04:53:34.529 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 112/171 2024-01-19T04:53:35.046 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 112/171 2024-01-19T04:53:35.082 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 113/171 2024-01-19T04:53:35.158 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 113/171 2024-01-19T04:53:35.158 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-19T04:53:35.158 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-19T04:53:35.158 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:35.192 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 114/171 2024-01-19T04:53:36.620 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 112/171 2024-01-19T04:53:36.654 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 112/171 2024-01-19T04:53:37.205 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 112/171 2024-01-19T04:53:37.241 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 113/171 2024-01-19T04:53:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 113/171 2024-01-19T04:53:37.342 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-19T04:53:37.342 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-19T04:53:37.342 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 114/171 2024-01-19T04:53:46.456 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 114/171 2024-01-19T04:53:46.456 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-01-19T04:53:46.457 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-01-19T04:53:46.457 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-01-19T04:53:46.457 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-01-19T04:53:46.457 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-01-19T04:53:46.457 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-01-19T04:53:46.457 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-01-19T04:53:46.457 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 115/171 2024-01-19T04:53:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 115/171 2024-01-19T04:53:46.630 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:46.630 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-19T04:53:46.630 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:48.109 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 116/171 2024-01-19T04:53:48.135 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 116/171 2024-01-19T04:53:48.135 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:48.135 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-19T04:53:48.135 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-19T04:53:48.135 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-19T04:53:48.135 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:48.360 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 117/171 2024-01-19T04:53:48.382 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 117/171 2024-01-19T04:53:48.383 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:48.383 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-19T04:53:48.383 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-19T04:53:48.383 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-19T04:53:48.383 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:48.827 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 118/171 2024-01-19T04:53:48.850 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 118/171 2024-01-19T04:53:48.850 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:48.850 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-19T04:53:48.850 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-19T04:53:48.850 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-19T04:53:48.850 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:48.936 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 119/171 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 114/171 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-01-19T04:53:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:49.316 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 120/171 2024-01-19T04:53:49.343 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 120/171 2024-01-19T04:53:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 115/171 2024-01-19T04:53:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-766.g0edf41a6.el 121/171 2024-01-19T04:53:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 115/171 2024-01-19T04:53:49.375 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:49.375 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-19T04:53:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-766.g0edf41a6.e 122/171 2024-01-19T04:53:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noar 123/171 2024-01-19T04:53:49.678 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noar 123/171 2024-01-19T04:53:49.709 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.no 124/171 2024-01-19T04:53:50.665 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.no 124/171 2024-01-19T04:53:50.692 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf 125/171 2024-01-19T04:53:50.792 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf 125/171 2024-01-19T04:53:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 116/171 2024-01-19T04:53:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 116/171 2024-01-19T04:53:50.922 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:50.922 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-19T04:53:50.922 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-19T04:53:50.923 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-19T04:53:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:50.954 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-766.g0edf41a6.el9 126/171 2024-01-19T04:53:50.986 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 127/171 2024-01-19T04:53:51.009 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 127/171 2024-01-19T04:53:51.010 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:51.010 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-19T04:53:51.010 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-19T04:53:51.010 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-19T04:53:51.011 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:51.054 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 128/171 2024-01-19T04:53:51.065 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 128/171 2024-01-19T04:53:51.114 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-766.g0edf41a6.el9.x86_64 129/171 2024-01-19T04:53:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 117/171 2024-01-19T04:53:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 117/171 2024-01-19T04:53:51.170 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:51.170 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-19T04:53:51.170 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-19T04:53:51.170 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-19T04:53:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:51.631 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 118/171 2024-01-19T04:53:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 118/171 2024-01-19T04:53:51.654 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:51.654 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-19T04:53:51.654 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-19T04:53:51.654 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-19T04:53:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:51.741 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 119/171 2024-01-19T04:53:51.894 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/171 2024-01-19T04:53:51.931 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 131/171 2024-01-19T04:53:51.953 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 131/171 2024-01-19T04:53:51.953 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:51.954 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-19T04:53:51.954 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-19T04:53:51.954 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-19T04:53:51.954 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:51.998 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-766.g0edf41 132/171 2024-01-19T04:53:52.020 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-766.g0edf41 132/171 2024-01-19T04:53:52.020 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:52.020 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-19T04:53:52.021 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:52.073 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 133/171 2024-01-19T04:53:52.096 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 133/171 2024-01-19T04:53:52.096 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:52.096 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-19T04:53:52.096 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-19T04:53:52.096 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-19T04:53:52.096 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 120/171 2024-01-19T04:53:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 120/171 2024-01-19T04:53:52.261 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-766.g0edf41a6.el 121/171 2024-01-19T04:53:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-766.g0edf41a6.e 122/171 2024-01-19T04:53:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noar 123/171 2024-01-19T04:53:52.399 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 134/171 2024-01-19T04:53:52.422 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 134/171 2024-01-19T04:53:52.422 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:52.422 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-19T04:53:52.423 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-19T04:53:52.423 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-19T04:53:52.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:53:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noar 123/171 2024-01-19T04:53:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.no 124/171 2024-01-19T04:53:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.no 124/171 2024-01-19T04:53:53.663 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf 125/171 2024-01-19T04:53:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf 125/171 2024-01-19T04:53:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-766.g0edf41a6.el9 126/171 2024-01-19T04:53:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 127/171 2024-01-19T04:53:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 127/171 2024-01-19T04:53:53.997 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:53.997 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-19T04:53:53.997 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-19T04:53:53.997 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-19T04:53:53.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 128/171 2024-01-19T04:53:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 128/171 2024-01-19T04:53:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.0.0-766.g0edf41a6.el9.x86_64 129/171 2024-01-19T04:53:54.943 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/171 2024-01-19T04:53:54.974 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 131/171 2024-01-19T04:53:54.997 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 131/171 2024-01-19T04:53:54.997 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:54.997 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-19T04:53:54.997 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-19T04:53:54.997 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-19T04:53:54.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-766.g0edf41 132/171 2024-01-19T04:53:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-766.g0edf41 132/171 2024-01-19T04:53:55.056 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:55.057 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-19T04:53:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 133/171 2024-01-19T04:53:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 133/171 2024-01-19T04:53:55.123 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:55.123 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-19T04:53:55.123 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-19T04:53:55.123 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-19T04:53:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:55.443 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 134/171 2024-01-19T04:53:55.465 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 134/171 2024-01-19T04:53:55.466 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-19T04:53:55.466 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-19T04:53:55.466 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-19T04:53:55.466 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-19T04:53:55.466 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:53:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-766.g0edf41a6.el9.x86_64 135/171 2024-01-19T04:53:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 136/171 2024-01-19T04:53:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-766.g0edf41a6.el9.x86_64 137/171 2024-01-19T04:53:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 138/171 2024-01-19T04:53:56.877 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/171 2024-01-19T04:53:57.069 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-766.g0edf41a6.el9.x86_64 140/171 2024-01-19T04:53:57.185 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 141/171 2024-01-19T04:53:57.216 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 142/171 2024-01-19T04:53:57.248 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-766.g0edf41a6.el9.x86_6 143/171 2024-01-19T04:53:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-19T04:53:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-19T04:53:57.382 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-19T04:53:57.436 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 147/171 2024-01-19T04:53:57.467 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 148/171 2024-01-19T04:53:58.088 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-19T04:53:58.124 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-19T04:53:58.167 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-19T04:53:58.222 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-19T04:53:59.026 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-766.g0edf41a6.el9.noarch 151/171 2024-01-19T04:53:59.087 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-19T04:53:59.128 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 153/171 2024-01-19T04:53:59.223 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 154/171 2024-01-19T04:53:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 155/171 2024-01-19T04:53:59.291 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-19T04:53:59.315 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-19T04:53:59.342 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-19T04:53:59.342 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-19T04:53:59.427 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-19T04:53:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 160/171 2024-01-19T04:53:59.486 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/171 2024-01-19T04:53:59.510 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/171 2024-01-19T04:53:59.535 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 163/171 2024-01-19T04:53:59.536 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-19T04:53:59.620 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-19T04:53:59.653 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-19T04:53:59.653 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-19T04:53:59.839 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-19T04:53:59.839 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-19T04:53:59.914 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-19T04:53:59.939 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-19T04:53:59.966 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-19T04:53:59.998 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-19T04:54:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.0.0-766.g0edf41a6.el9.x86_64 135/171 2024-01-19T04:54:00.063 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 136/171 2024-01-19T04:54:00.063 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-19T04:54:00.104 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.0.0-766.g0edf41a6.el9.x86_64 137/171 2024-01-19T04:54:00.135 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 138/171 2024-01-19T04:54:00.162 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/171 2024-01-19T04:54:00.370 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.0.0-766.g0edf41a6.el9.x86_64 140/171 2024-01-19T04:54:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 141/171 2024-01-19T04:54:00.517 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 142/171 2024-01-19T04:54:00.550 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.0.0-766.g0edf41a6.el9.x86_6 143/171 2024-01-19T04:54:00.582 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-19T04:54:00.612 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-19T04:54:00.703 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-19T04:54:00.759 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 147/171 2024-01-19T04:54:00.820 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 148/171 2024-01-19T04:54:01.408 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-19T04:54:01.442 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-19T04:54:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-19T04:54:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.0.0-766.g0edf41a6.el9.noarch 151/171 2024-01-19T04:54:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-19T04:54:02.521 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 153/171 2024-01-19T04:54:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 154/171 2024-01-19T04:54:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 155/171 2024-01-19T04:54:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-19T04:54:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-19T04:54:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-19T04:54:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-19T04:54:02.819 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-19T04:54:02.864 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 160/171 2024-01-19T04:54:02.895 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/171 2024-01-19T04:54:02.919 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/171 2024-01-19T04:54:02.944 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 163/171 2024-01-19T04:54:02.945 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-19T04:54:03.029 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-19T04:54:03.054 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-19T04:54:03.062 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-19T04:54:03.062 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-19T04:54:03.248 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-19T04:54:03.249 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-19T04:54:03.324 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-19T04:54:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-19T04:54:03.475 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-766.g0edf41a6.el9.x86_64 1/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 2/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 3/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 4/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-766.g0edf41 5/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 6/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 7/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 8/171 2024-01-19T04:54:04.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 9/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 10/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 11/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-766.g0edf41a6.el9.x86_64 12/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 13/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-766.g0edf41a6.el9.x86_64 14/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-766.g0edf41a6.el9.x86_6 15/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 16/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 17/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 18/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_ 19/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 20/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-766.g0edf41a6.el9 21/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-766.g0edf41a6.el9.x 22/171 2024-01-19T04:54:04.902 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-766.g0edf41a6.el9.x86_64 23/171 2024-01-19T04:54:04.903 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-766.g0edf41a6.el9.x86_64 24/171 2024-01-19T04:54:04.903 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-766.g0edf41a6.el9.x86_64 25/171 2024-01-19T04:54:04.903 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-766.g0edf41a6.el9.x86_64 26/171 2024-01-19T04:54:04.903 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 27/171 2024-01-19T04:54:04.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 28/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-766.g0edf41a6.el9.x86_64 29/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-766.g0edf41a6.e 30/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noar 31/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.no 32/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf 33/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-766.g0edf41a6.el9 34/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 35/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-766.g0edf41a6.el 36/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 37/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 38/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-766.g0edf41a6.el9.noarch 39/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/171 2024-01-19T04:54:04.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/171 2024-01-19T04:54:04.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 64/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 65/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 66/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 67/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 68/171 2024-01-19T04:54:04.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 69/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 70/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 71/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 72/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 73/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 74/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/171 2024-01-19T04:54:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 84/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 85/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 86/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 87/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-16.el9.x86_64 88/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 91/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 92/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 93/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 94/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 95/171 2024-01-19T04:54:04.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 96/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 97/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 98/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-19T04:54:04.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 115/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-19T04:54:04.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 140/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 142/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 144/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 145/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 146/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 147/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 148/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 150/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 151/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 152/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 153/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-4.el9.x86_64 154/171 2024-01-19T04:54:04.925 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-3.el9.x86_64 155/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 156/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 158/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 159/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 160/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 161/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 162/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 163/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 164/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 165/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 166/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 167/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 168/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 169/171 2024-01-19T04:54:04.926 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 170/171 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 171/171 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: acl-2.3.1-4.el9.x86_64 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-19T04:54:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-19T04:54:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-16.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-19T04:54:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-19T04:54:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.11-1.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-19T04:54:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-19T04:54:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:07.755 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-19T04:54:08.170 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-19T04:54:08.170 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.0.0-766.g0edf41a6.el9.x86_64 1/171 2024-01-19T04:54:08.170 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 2/171 2024-01-19T04:54:08.170 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 3/171 2024-01-19T04:54:08.170 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 4/171 2024-01-19T04:54:08.170 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-766.g0edf41 5/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 6/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 7/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 8/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 9/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 10/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 11/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.0.0-766.g0edf41a6.el9.x86_64 12/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 13/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.0.0-766.g0edf41a6.el9.x86_64 14/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.0.0-766.g0edf41a6.el9.x86_6 15/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 16/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 17/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 18/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_ 19/171 2024-01-19T04:54:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 20/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.0.0-766.g0edf41a6.el9 21/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.0.0-766.g0edf41a6.el9.x 22/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.0.0-766.g0edf41a6.el9.x86_64 23/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.0.0-766.g0edf41a6.el9.x86_64 24/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.0.0-766.g0edf41a6.el9.x86_64 25/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.0.0-766.g0edf41a6.el9.x86_64 26/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 27/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 28/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.0.0-766.g0edf41a6.el9.x86_64 29/171 2024-01-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-766.g0edf41a6.e 30/171 2024-01-19T04:54:08.184 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noar 31/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.no 32/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf 33/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-766.g0edf41a6.el9 34/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 35/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-766.g0edf41a6.el 36/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 37/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 38/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.0.0-766.g0edf41a6.el9.noarch 39/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/171 2024-01-19T04:54:08.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/171 2024-01-19T04:54:08.186 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 64/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 65/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 66/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 67/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 68/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 69/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 70/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 71/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 72/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 73/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 74/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/171 2024-01-19T04:54:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 84/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 85/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 86/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 87/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.6-16.el9.x86_64 88/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/171 2024-01-19T04:54:08.188 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 91/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 92/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 93/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 94/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 95/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 96/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 97/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 98/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-19T04:54:08.189 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 115/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-19T04:54:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-19T04:54:08.191 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 140/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 142/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 144/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 145/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 146/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 147/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 148/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/171 2024-01-19T04:54:08.192 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 150/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 151/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 152/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 153/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : acl-2.3.1-4.el9.x86_64 154/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : acl-2.3.1-3.el9.x86_64 155/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 156/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 158/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 159/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 160/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 161/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 162/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 163/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 164/171 2024-01-19T04:54:08.193 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 165/171 2024-01-19T04:54:08.194 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 166/171 2024-01-19T04:54:08.194 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 167/171 2024-01-19T04:54:08.194 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 168/171 2024-01-19T04:54:08.194 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 169/171 2024-01-19T04:54:08.194 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 170/171 2024-01-19T04:54:08.291 DEBUG:teuthology.parallel:result is None 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 171/171 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: acl-2.3.1-4.el9.x86_64 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-19T04:54:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-19T04:54:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.0.0-766.g0edf41a6.el9.noarch 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-19T04:54:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.6-16.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-19T04:54:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-19T04:54:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.11-1.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-19T04:54:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-19T04:54:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-19T04:54:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-19T04:54:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.0.0-766.g0edf41a6.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-3.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-19T04:54:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-19T04:54:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:11.179 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-19T04:54:11.751 DEBUG:teuthology.parallel:result is None 2024-01-19T04:54:11.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0edf41a622739843d6b978b179ff3227b476dd9d 2024-01-19T04:54:11.896 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-19T04:54:11.918 INFO:teuthology.orchestra.run.smithi031.stdout:19.0.0-766.g0edf41a6.el9 2024-01-19T04:54:11.919 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-766.g0edf41a6.el9 2024-01-19T04:54:11.919 INFO:teuthology.task.install:The correct ceph version 19.0.0-766.g0edf41a6 is installed. 2024-01-19T04:54:11.921 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0edf41a622739843d6b978b179ff3227b476dd9d 2024-01-19T04:54:12.092 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-19T04:54:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-766.g0edf41a6.el9 2024-01-19T04:54:12.118 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-766.g0edf41a6.el9 2024-01-19T04:54:12.118 INFO:teuthology.task.install:The correct ceph version 19.0.0-766.g0edf41a6 is installed. 2024-01-19T04:54:12.119 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-19T04:54:12.120 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:12.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-19T04:54:12.151 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:12.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-19T04:54:12.193 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-19T04:54:12.193 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:12.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-01-19T04:54:12.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-19T04:54:12.289 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:12.290 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-01-19T04:54:12.321 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-19T04:54:12.391 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-19T04:54:12.391 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:12.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-19T04:54:12.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-19T04:54:12.486 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:12.486 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-19T04:54:12.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-19T04:54:12.584 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-19T04:54:12.584 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:12.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-01-19T04:54:12.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-19T04:54:12.678 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:12.678 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-01-19T04:54:12.707 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-19T04:54:12.775 INFO:teuthology.run_tasks:Running task ceph... 2024-01-19T04:54:12.855 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-19T04:54:12.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-01-19T04:54:12.858 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /var/log/ceph 2024-01-19T04:54:12.886 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-19T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-19T04:54:12.925 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-19T04:54:12.953 INFO:tasks.ceph:Creating extra log directories... 2024-01-19T04:54:12.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-19T04:54:12.991 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-19T04:54:13.029 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-19T04:54:13.029 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-19T04:54:13.030 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-19T04:54:13.030 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:13.030 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-19T04:54:13.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-19T04:54:13.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-19T04:54:13.214 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:13.215 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-19T04:54:13.230 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-19T04:54:13.352 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-19T04:54:13.385 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-19T04:54:13.385 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-19T04:54:13.385 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038', 'branch': 'wip-yuri8-testing-2024-01-18-0823', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7521038', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri8-testing-2024-01-18-0823'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri8-testing-2024-01-18-0823', 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6631, 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri8-testing-2024-01-18-0823', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '0edf41a622739843d6b978b179ff3227b476dd9d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfCsrSuIklKEIe2LJNcxUOdkdiyQIl89QsL9zj3TrLRIJqWDvPNGcqZ4/uam4JLHEob6pP0RSQxbCMD2dg06FQ=', 'smithi137.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNazwqQDaviTxfMbB+N/FPtO/PTb6ysR06SdlXD5H6640Cm2XYEZznkQmi/ZEo90JW1vpWugaybplenJ7ZsRFEw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '0edf41a622739843d6b978b179ff3227b476dd9d', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-18_21:19:14', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-19T04:54:13.385 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-19T04:54:13.387 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-19T04:54:13.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-01-19T04:54:13.441 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/run/ceph 2024-01-19T04:54:13.502 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:13.502 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-01-19T04:54:13.520 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-19T04:54:13.520 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-01-19T04:54:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-19T04:54:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:13.574 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-01-19T04:54:13.574 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:13.575 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:13.575 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-19 04:54:03.821895954 +0000 2024-01-19T04:54:13.575 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-19 04:52:10.429751100 +0000 2024-01-19T04:54:13.575 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-19 04:52:10.429751100 +0000 2024-01-19T04:54:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-19 04:52:10.429751100 +0000 2024-01-19T04:54:13.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-19T04:54:13.638 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-19T04:54:13.639 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-19T04:54:13.639 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000253798 s, 2.0 MB/s 2024-01-19T04:54:13.640 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-19T04:54:13.695 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-01-19T04:54:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-19T04:54:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:13.749 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-01-19T04:54:13.749 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:13.749 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:13.750 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-19 04:54:03.821895954 +0000 2024-01-19T04:54:13.750 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-19 04:52:10.777745294 +0000 2024-01-19T04:54:13.750 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-19 04:52:10.777745294 +0000 2024-01-19T04:54:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-19 04:52:10.777745294 +0000 2024-01-19T04:54:13.750 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-19T04:54:13.814 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-19T04:54:13.814 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-19T04:54:13.814 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000178685 s, 2.9 MB/s 2024-01-19T04:54:13.815 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-19T04:54:13.870 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-01-19T04:54:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-19T04:54:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:13.924 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-01-19T04:54:13.924 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:13.924 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:13.925 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-19 04:54:03.822895938 +0000 2024-01-19T04:54:13.925 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-19 04:52:11.121739555 +0000 2024-01-19T04:54:13.925 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-19 04:52:11.121739555 +0000 2024-01-19T04:54:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-19 04:52:11.121739555 +0000 2024-01-19T04:54:13.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-19T04:54:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-19T04:54:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-19T04:54:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000204438 s, 2.5 MB/s 2024-01-19T04:54:13.990 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-19T04:54:14.047 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-01-19T04:54:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-19T04:54:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-01-19T04:54:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-19 04:54:03.822895938 +0000 2024-01-19T04:54:14.102 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-19 04:52:11.476733632 +0000 2024-01-19T04:54:14.102 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-19 04:52:11.476733632 +0000 2024-01-19T04:54:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-19 04:52:11.476733632 +0000 2024-01-19T04:54:14.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-19T04:54:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-19T04:54:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-19T04:54:14.166 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000170127 s, 3.0 MB/s 2024-01-19T04:54:14.167 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-19T04:54:14.222 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:14.222 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:14.222 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-01-19T04:54:14.239 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-19T04:54:14.239 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-01-19T04:54:14.295 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-19 04:54:00.385492279 +0000 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-19 04:52:08.516618109 +0000 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-19 04:52:08.516618109 +0000 2024-01-19T04:54:14.296 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-19 04:52:08.516618109 +0000 2024-01-19T04:54:14.296 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-19T04:54:14.364 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-19T04:54:14.364 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-19T04:54:14.364 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000212756 s, 2.4 MB/s 2024-01-19T04:54:14.365 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-19T04:54:14.424 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-01-19T04:54:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-19T04:54:14.488 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:14.488 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-01-19T04:54:14.489 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:14.489 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:14.489 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-19 04:54:00.385492279 +0000 2024-01-19T04:54:14.489 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-19 04:52:08.852612194 +0000 2024-01-19T04:54:14.489 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-19 04:52:08.852612194 +0000 2024-01-19T04:54:14.489 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-19 04:52:08.852612194 +0000 2024-01-19T04:54:14.490 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-19T04:54:14.572 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-19T04:54:14.572 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-19T04:54:14.572 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000194164 s, 2.6 MB/s 2024-01-19T04:54:14.573 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-19T04:54:14.631 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-01-19T04:54:14.693 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-19 04:54:00.386492262 +0000 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-19 04:52:09.190606244 +0000 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-19 04:52:09.190606244 +0000 2024-01-19T04:54:14.694 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-19 04:52:09.190606244 +0000 2024-01-19T04:54:14.694 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-19T04:54:14.765 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-19T04:54:14.765 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-19T04:54:14.765 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000197627 s, 2.6 MB/s 2024-01-19T04:54:14.766 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-19T04:54:14.828 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-01-19T04:54:14.889 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-19T04:54:14.889 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-19T04:54:14.890 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-01-19T04:54:14.890 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-19T04:54:14.890 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-19T04:54:14.890 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-19 04:54:00.386492262 +0000 2024-01-19T04:54:14.890 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-19 04:52:09.525600346 +0000 2024-01-19T04:54:14.890 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-19 04:52:09.525600346 +0000 2024-01-19T04:54:14.890 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-19 04:52:09.525600346 +0000 2024-01-19T04:54:14.891 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-19T04:54:14.968 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-19T04:54:14.968 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-19T04:54:14.968 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000187925 s, 2.7 MB/s 2024-01-19T04:54:14.969 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-19T04:54:15.027 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:15.027 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-19T04:54:15.027 INFO:tasks.ceph:Generating config... 2024-01-19T04:54:15.028 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-19T04:54:15.028 INFO:tasks.ceph:[client] debug client = 20 2024-01-19T04:54:15.028 INFO:tasks.ceph:[client] debug ms = 1 2024-01-19T04:54:15.028 INFO:tasks.ceph:[client] pid file = 2024-01-19T04:54:15.029 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-19T04:54:15.029 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-19T04:54:15.029 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-19T04:54:15.030 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-19T04:54:15.030 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-19T04:54:15.030 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-19T04:54:15.030 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-19T04:54:15.030 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-19T04:54:15.031 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-19T04:54:15.031 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-19T04:54:15.031 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-19T04:54:15.031 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-19T04:54:15.031 INFO:tasks.ceph:Setting up mon.a... 2024-01-19T04:54:15.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-19T04:54:15.091 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-01-19T04:54:15.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-19T04:54:15.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-19T04:54:15.173 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.b', '172.21.15.137'), ('mon.c', '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]')] 2024-01-19T04:54:15.174 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-19T04:54:15.175 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:15.175 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-19T04:54:15.232 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --add b 172.21.15.137 --addv c '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 16b8b5c3-25df-408f-9c77-cd32e346926d 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 16b8b5c3-25df-408f-9c77-cd32e346926d 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-01-19T04:54:15.322540+0000 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-01-19T04:54:15.322540+0000 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 17 (quincy) 2024-01-19T04:54:15.321 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-01-19T04:54:15.322 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-01-19T04:54:15.322 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-01-19T04:54:15.322 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-01-19T04:54:15.322 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-19T04:54:15.323 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-19T04:54:15.338 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 16b8b5c3-25df-408f-9c77-cd32e346926d... 2024-01-19T04:54:15.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-19T04:54:15.382 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-19T04:54:15.430 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-01-19T04:54:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-01-19T04:54:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-01-19T04:54:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-01-19T04:54:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-19T04:54:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-01-19T04:54:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-01-19T04:54:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-01-19T04:54:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-01-19T04:54:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-01-19T04:54:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-01-19T04:54:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-01-19T04:54:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-01-19T04:54:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-19T04:54:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-19T04:54:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-01-19T04:54:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-19T04:54:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-01-19T04:54:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-01-19T04:54:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-01-19T04:54:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-01-19T04:54:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-01-19T04:54:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]" 2024-01-19T04:54:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 16b8b5c3-25df-408f-9c77-cd32e346926d 2024-01-19T04:54:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.438 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-01-19T04:54:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-01-19T04:54:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-01-19T04:54:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-01-19T04:54:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-01-19T04:54:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-01-19T04:54:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-01-19T04:54:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-01-19T04:54:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-01-19T04:54:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-01-19T04:54:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: bdev async discard = True 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: bdev enable discard = True 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore allocator = bitmap 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2024-01-19T04:54:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-01-19T04:54:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-01-19T04:54:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-19T04:54:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-01-19T04:54:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: mon op complaint time = 120 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-01-19T04:54:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: client mount timeout = 600 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = "" 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2024-01-19T04:54:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout:[mds] 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds = 20 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds balancer = 20 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug frag = True 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug scatterstat = True 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: mds op complaint time = 180 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: mds verify scatter = True 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2024-01-19T04:54:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: chdir = "" 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: auth supported = cephx 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: filestore xattr use omap = true 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon clock drift allowed = 1.000 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.449 INFO:teuthology.orchestra.run.smithi137.stdout: osd crush chooseleaf type = 0 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: auth debug = true 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on old message = true 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on bug = true 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon pg warn max object skew = 0 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: # disable pg_autoscaler by default for new pools 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default size = 2 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow primary affinity = true 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow pg remap = true 2024-01-19T04:54:15.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on legacy crush tunables = false 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on crush straw calc version zero = false 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on no sortbitwise = false 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on osd down out interval zero = false 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on too few osds = false 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon_allow_pool_size_one = true 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: osd default data pool replay window = 5 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon allow pool delete = true 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon cluster log file level = debug 2024-01-19T04:54:15.451 INFO:teuthology.orchestra.run.smithi137.stdout: debug asserts on shutdown = true 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: mon health detail to clog = false 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: mon host = "172.21.15.31,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]" 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 16b8b5c3-25df-408f-9c77-cd32e346926d 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout:[osd] 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: osd journal size = 100 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub load threshold = 5.0 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub max interval = 600 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: osd mclock profile = high_recovery_ops 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: osd recover clone overlap = true 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: osd recovery max chunk = 1048576 2024-01-19T04:54:15.452 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug shutdown = true 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug op order = true 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify stray on activate = true 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug trim objects = true 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd open classes on start = true 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug pg log writeout = true 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd deep scrub update digest min age = 30 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: osd map max advance = 10 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: journal zero on create = true 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: filestore ondisk finisher threads = 3 2024-01-19T04:54:15.453 INFO:teuthology.orchestra.run.smithi137.stdout: filestore apply finisher threads = 3 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: bdev debug aio = true 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug misdirected ops = true 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: bdev async discard = True 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: bdev enable discard = True 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore allocator = bitmap 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore block size = 96636764160 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore fsck on mount = True 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluefs = 1/20 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluestore = 1/20 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: debug osd = 20 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: debug rocksdb = 4/10 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd backfillfull_ratio = 0.85 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd full ratio = 0.9 2024-01-19T04:54:15.454 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd nearfull ratio = 0.8 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: osd failsafe full ratio = 0.95 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: osd objectstore = bluestore 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: osd op complaint time = 180 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr] 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: debug mgr = 20 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: mgr/telemetry/nag = false 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout:[mon] 2024-01-19T04:54:15.455 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: debug paxos = 20 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: mon data avail warn = 5 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: mon mgr mkfs grace = 240 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd reporter subtree level = osd 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd prime pg temp = true 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: auth mon ticket ttl = 660 # 11m 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: auth service ticket ttl = 240 # 4m 2024-01-19T04:54:15.456 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: # don't complain about insecure global_id in the test suite 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: # 1m isn't quite enough 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: mon_down_mkfs_grace = 2m 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_filestore_osds = false 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: mon op complaint time = 120 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout:[client] 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: rgw cache enabled = true 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable ops log = true 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable usage log = true 2024-01-19T04:54:15.457 INFO:teuthology.orchestra.run.smithi137.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: client mount timeout = 600 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: debug client = 20 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: pid file = "" 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: rados mon op timeout = 900 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: rados osd op timeout = 900 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.a] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.b] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.c] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout:[mds] 2024-01-19T04:54:15.458 INFO:teuthology.orchestra.run.smithi137.stdout: debug mds = 20 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: debug mds balancer = 20 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: mds debug frag = True 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: mds debug scatterstat = True 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: mds op complaint time = 180 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: mds verify scatter = True 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: osd op complaint time = 180 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: rados mon op timeout = 900 2024-01-19T04:54:15.459 INFO:teuthology.orchestra.run.smithi137.stdout: rados osd op timeout = 900 2024-01-19T04:54:15.459 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-19T04:54:15.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-19T04:54:15.544 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-19T04:54:15.544 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:15.544 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-19T04:54:15.558 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:15.558 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-19T04:54:15.612 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:54:15.613 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:15.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-19T04:54:15.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-19T04:54:15.692 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:15.692 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:15.746 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:54:15.746 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:15.746 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-19T04:54:15.746 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-19T04:54:15.786 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:15.787 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:15.842 INFO:tasks.ceph:Setting up mon nodes... 2024-01-19T04:54:15.843 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-19T04:54:15.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-19T04:54:15.897 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-19T04:54:15.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-19T04:54:15.961 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-19T04:54:15.964 INFO:tasks.ceph:Setting up mds nodes... 2024-01-19T04:54:15.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-19T04:54:16.018 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-19T04:54:16.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-19T04:54:16.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-19T04:54:16.141 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-19T04:54:16.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-19T04:54:16.170 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-19T04:54:16.225 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-19T04:54:16.228 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-19T04:54:16.254 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-19T04:54:16.348 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-19T04:54:16.351 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-19T04:54:16.377 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-19T04:54:16.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-19T04:54:16.419 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-19T04:54:16.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-19T04:54:16.516 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-19T04:54:16.532 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-19T04:54:16.573 INFO:teuthology.orchestra.run.smithi137.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-19T04:54:16.588 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-19T04:54:16.671 INFO:teuthology.orchestra.run.smithi137.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-19T04:54:16.686 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-19T04:54:16.686 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-19T04:54:16.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-19T04:54:16.712 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:16.713 INFO:tasks.ceph:role: osd.0 2024-01-19T04:54:16.713 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:54:16.713 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-19T04:54:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:17.275 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-01-19T04:54:17.277 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-01-19T04:54:17.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-19T04:54:17.418 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-19T04:54:17.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-19T04:54:17.520 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:17.520 INFO:tasks.ceph:role: osd.1 2024-01-19T04:54:17.520 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:54:17.521 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:17.595 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:18.056 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-01-19T04:54:18.058 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-01-19T04:54:18.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-19T04:54:18.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-19T04:54:18.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-19T04:54:18.255 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:18.255 INFO:tasks.ceph:role: osd.2 2024-01-19T04:54:18.255 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:54:18.255 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:18.331 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-01-19T04:54:18.827 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-01-19T04:54:18.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-19T04:54:18.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-19T04:54:18.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-19T04:54:19.063 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:19.063 INFO:tasks.ceph:role: osd.3 2024-01-19T04:54:19.063 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:54:19.063 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-19T04:54:19.140 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:19.140 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:19.141 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:19.141 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:19.141 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:19.653 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-01-19T04:54:19.655 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-01-19T04:54:19.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-19T04:54:19.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-19T04:54:19.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:19.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:19.899+0000 7f4477dfe4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-19T04:54:19.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:19.899+0000 7f4477dfe4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-19T04:54:19.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:19.899+0000 7f4477dfe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-19T04:54:19.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:19.899+0000 7f4477dfe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-19T04:54:19.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:19.899+0000 7f4477dfe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-19T04:54:19.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:19.900+0000 7f4477dfe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-19T04:54:22.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-19T04:54:22.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:22.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:22.875+0000 7f74ae6124c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-19T04:54:22.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:22.875+0000 7f74ae6124c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-19T04:54:22.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:22.875+0000 7f74ae6124c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-19T04:54:22.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:22.875+0000 7f74ae6124c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-19T04:54:22.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:22.875+0000 7f74ae6124c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-19T04:54:22.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:22.876+0000 7f74ae6124c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-19T04:54:25.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-19T04:54:25.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:25.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:25.848+0000 7f12428224c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-19T04:54:25.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:25.848+0000 7f12428224c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-19T04:54:25.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:25.848+0000 7f12428224c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-19T04:54:25.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:25.848+0000 7f12428224c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-19T04:54:25.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:25.848+0000 7f12428224c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-19T04:54:25.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:25.849+0000 7f12428224c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-19T04:54:28.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-19T04:54:28.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:28.820+0000 7f46120104c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-19T04:54:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:28.820+0000 7f46120104c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-19T04:54:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:28.820+0000 7f46120104c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-19T04:54:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:28.820+0000 7f46120104c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-19T04:54:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:28.820+0000 7f46120104c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-19T04:54:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:28.821+0000 7f46120104c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-19T04:54:31.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-19T04:54:31.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-19T04:54:31.730 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:31.730 INFO:tasks.ceph:role: osd.4 2024-01-19T04:54:31.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:54:31.730 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-19T04:54:31.809 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:31.809 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:31.810 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:32.325 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-01-19T04:54:32.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-01-19T04:54:32.327 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-19T04:54:32.473 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-19T04:54:32.509 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-19T04:54:32.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:32.576 INFO:tasks.ceph:role: osd.5 2024-01-19T04:54:32.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:54:32.576 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-19T04:54:32.651 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:32.651 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:32.651 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:32.651 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:32.651 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:32.652 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:32.652 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:32.652 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:32.652 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:32.652 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:33.100 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-01-19T04:54:33.102 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-01-19T04:54:33.102 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-19T04:54:33.193 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-19T04:54:33.225 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-19T04:54:33.292 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:33.292 INFO:tasks.ceph:role: osd.6 2024-01-19T04:54:33.292 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:54:33.293 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-19T04:54:33.366 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:33.366 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:33.366 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:33.366 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:33.366 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:33.366 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:33.367 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:33.367 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:33.367 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:33.367 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:33.853 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-01-19T04:54:33.854 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-01-19T04:54:33.854 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-19T04:54:33.984 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-19T04:54:34.018 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-19T04:54:34.085 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-19T04:54:34.085 INFO:tasks.ceph:role: osd.7 2024-01-19T04:54:34.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:54:34.085 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-19T04:54:34.160 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-19T04:54:34.161 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-19T04:54:34.161 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-19T04:54:34.663 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-01-19T04:54:34.665 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-01-19T04:54:34.665 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-19T04:54:34.794 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-19T04:54:34.827 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:34.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:34.928+0000 7f41e340e4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-19T04:54:34.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:34.928+0000 7f41e340e4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-19T04:54:34.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:34.928+0000 7f41e340e4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-19T04:54:34.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:34.928+0000 7f41e340e4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-19T04:54:34.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:34.928+0000 7f41e340e4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-19T04:54:34.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:34.929+0000 7f41e340e4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-19T04:54:37.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-19T04:54:37.818 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:37.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:37.909+0000 7efd96a0d4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-19T04:54:37.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:37.909+0000 7efd96a0d4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-19T04:54:37.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:37.909+0000 7efd96a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-19T04:54:37.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:37.909+0000 7efd96a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-19T04:54:37.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:37.909+0000 7efd96a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-19T04:54:37.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:37.910+0000 7efd96a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-19T04:54:40.761 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-19T04:54:40.792 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:40.883+0000 7fb2cd01d4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-19T04:54:40.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:40.883+0000 7fb2cd01d4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-19T04:54:40.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:40.883+0000 7fb2cd01d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-19T04:54:40.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:40.883+0000 7fb2cd01d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-19T04:54:40.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:40.883+0000 7fb2cd01d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-19T04:54:40.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:40.884+0000 7fb2cd01d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-19T04:54:43.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:54:43.390 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:54:43.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-19T04:54:43.754 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:43.840+0000 7f482d6fa4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-19T04:54:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:43.840+0000 7f482d6fa4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-19T04:54:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:43.840+0000 7f482d6fa4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-19T04:54:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:43.840+0000 7f482d6fa4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-19T04:54:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:43.840+0000 7f482d6fa4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-19T04:54:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:43.841+0000 7f482d6fa4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-19T04:54:46.685 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-19T04:54:46.716 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-19T04:54:46.716 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:46.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-19T04:54:46.747 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:46.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-19T04:54:46.817 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:46.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-19T04:54:46.887 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:46.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-19T04:54:46.956 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:46.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-19T04:54:47.024 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:47.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-19T04:54:47.092 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:47.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-19T04:54:47.160 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-19T04:54:47.192 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.192 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-19T04:54:47.263 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.263 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-19T04:54:47.333 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.333 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-19T04:54:47.402 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.402 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-19T04:54:47.471 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.471 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-19T04:54:47.539 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.539 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-19T04:54:47.607 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:47.607 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-19T04:54:47.623 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:47.624 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-19T04:54:47.679 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.679 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-19T04:54:47.695 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:47.695 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-19T04:54:47.750 INFO:tasks.ceph:Adding keys to all mons... 2024-01-19T04:54:47.750 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-19T04:54:47.752 DEBUG:teuthology.orchestra.run.smithi137:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-19T04:54:47.793 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB3AKpl6LqONRAAoCHvKhaqK1R2nkcFvmGb1A== 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKpl9vwvARAAZduX/PeYlqDH8VDbvZiELg== 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplMbeJCBAAkHOlhUk5I9r4TWFRo9L/eA== 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB7AKplbqKsNRAAikErjUtmLKvlkeH7GGL1UQ== 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB+AKplVZk5NBAA6FIuSbDo0CCaftcB9zQbmw== 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCBAKplP8CiMhAABJdd6kUn7DlZFCikXiggGQ== 2024-01-19T04:54:47.794 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCEAKplPAbxMBAANYNlBOGgzR60cDNfMpZTDA== 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB3AKplwl9qORAAdjRFheF+sxCmZe31mpCjVw== 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKpl6TSLDRAAGB2c1AX6e5HAuMDl6tCbKw== 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplQl/gFBAA3VbnYQWE/2FHLv7opt1f9w== 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCKAKplzlV2NxAAsprGSRQCPR98cdSNEBqmqA== 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCNAKplqnZRNhAA2gX5iijb/OorAr9M0kxMBw== 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCQAKplG4yxNBAAIgP3l7C0BuCHY3ZzK0ezuQ== 2024-01-19T04:54:47.795 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCTAKplKnckMhAApf2mP+F7NZ39VBHDkNVBTg== 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout:[client.2] 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKpl9grnHhAAnUsK7YtYaC8ZPOX4W84PbA== 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKpl11pPIhAAr+yfn0112ldu1NcYFzRIZg== 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout:[client.3] 2024-01-19T04:54:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplog8YKBAA9SYGPnH5B4/9JInpxCxziw== 2024-01-19T04:54:47.816 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.y] 2024-01-19T04:54:47.816 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB3AKpl6LqONRAAoCHvKhaqK1R2nkcFvmGb1A== 2024-01-19T04:54:47.816 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a] 2024-01-19T04:54:47.816 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKpl9vwvARAAZduX/PeYlqDH8VDbvZiELg== 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c] 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKplMbeJCBAAkHOlhUk5I9r4TWFRo9L/eA== 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.0] 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB7AKplbqKsNRAAikErjUtmLKvlkeH7GGL1UQ== 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.1] 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB+AKplVZk5NBAA6FIuSbDo0CCaftcB9zQbmw== 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.2] 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCBAKplP8CiMhAABJdd6kUn7DlZFCikXiggGQ== 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.3] 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCEAKplPAbxMBAANYNlBOGgzR60cDNfMpZTDA== 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.x] 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB3AKplwl9qORAAdjRFheF+sxCmZe31mpCjVw== 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b] 2024-01-19T04:54:47.817 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKpl6TSLDRAAGB2c1AX6e5HAuMDl6tCbKw== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKplQl/gFBAA3VbnYQWE/2FHLv7opt1f9w== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.4] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCKAKplzlV2NxAAsprGSRQCPR98cdSNEBqmqA== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.5] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCNAKplqnZRNhAA2gX5iijb/OorAr9M0kxMBw== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.6] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCQAKplG4yxNBAAIgP3l7C0BuCHY3ZzK0ezuQ== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.7] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCTAKplKnckMhAApf2mP+F7NZ39VBHDkNVBTg== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[client.2] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKpl9grnHhAAnUsK7YtYaC8ZPOX4W84PbA== 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout:[client.1] 2024-01-19T04:54:47.818 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKpl11pPIhAAr+yfn0112ldu1NcYFzRIZg== 2024-01-19T04:54:47.819 INFO:teuthology.orchestra.run.smithi137.stdout:[client.3] 2024-01-19T04:54:47.819 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB4AKplog8YKBAA9SYGPnH5B4/9JInpxCxziw== 2024-01-19T04:54:47.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-19T04:54:47.840 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-19T04:54:47.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:47.915 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:47.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:47.972 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:48.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.028 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.082 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.136 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.190 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-19T04:54:48.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-19T04:54:48.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:48.296 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:48.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:48.353 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-19T04:54:48.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.410 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.465 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.519 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.572 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-19T04:54:48.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.626 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.682 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.739 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.796 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-19T04:54:48.850 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-19T04:54:48.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-19T04:54:48.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-19T04:54:48.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-19T04:54:49.000 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-19T04:54:49.027 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-19T04:54:49.123 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-19T04:54:49.150 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-19T04:54:49.217 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-19T04:54:49.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-19T04:54:49.341 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:49.343 DEBUG:teuthology.orchestra.run.smithi137:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-19T04:54:49.395 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-19T04:54:49.396 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-19T04:54:49.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-19T04:54:49.398 INFO:tasks.ceph.mon.a:Started 2024-01-19T04:54:49.398 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-19T04:54:49.398 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-19T04:54:49.437 INFO:tasks.ceph.mon.b:Started 2024-01-19T04:54:49.437 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-19T04:54:49.438 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-19T04:54:49.439 INFO:tasks.ceph.mon.c:Started 2024-01-19T04:54:49.439 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-19T04:54:49.440 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-19T04:54:49.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-19T04:54:49.442 INFO:tasks.ceph.mgr.y:Started 2024-01-19T04:54:49.442 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-19T04:54:49.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-19T04:54:49.444 INFO:tasks.ceph.mgr.x:Started 2024-01-19T04:54:49.444 INFO:tasks.ceph:Setting crush tunables to default 2024-01-19T04:54:49.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-01-19T04:54:49.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.546+0000 7f4022eb4640 1 Processor -- start 2024-01-19T04:54:49.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.546+0000 7f4022eb4640 1 -- start start 2024-01-19T04:54:49.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f4022eb4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:49.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f4022eb4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:49.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f4022eb4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c145080 con 0x7f401c15fc50 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f4022eb4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c0722c0 con 0x7f401c15c3a0 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f4022eb4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c15f280 con 0x7f401c144ca0 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f401bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f401bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44486/0 (socket says 172.21.15.31:44486) 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 learned_addr learned my addr 172.21.15.31:0/2962617155 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.547+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.548+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.548+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.548+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:49.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.548+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-19T04:54:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.748+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.748+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.748+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.748+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.748+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-19T04:54:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.749+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.749+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.749+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.749+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:49.749+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-19T04:54:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.149+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.149+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.149+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.149+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.149+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-19T04:54:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.150+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.150+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.150+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.150+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.150+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-19T04:54:50.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.950+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.950+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.950+0000 7f401bfff640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.950+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.950+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-19T04:54:50.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.951+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:50.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.951+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:50.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.951+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:50.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.951+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:50.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:50.951+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-19T04:54:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 msgr2=0x7f401c071ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-19T04:54:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.137:3300/0 conn(0x7f401c144ca0 0x7f401c071ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 -- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 msgr2=0x7f401c15f710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 --2- 172.21.15.31:0/2962617155 >> v2:172.21.15.31:3300/0 conn(0x7f401c15c3a0 0x7f401c15f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 -- 172.21.15.31:0/2962617155 >> v1:172.21.15.137:6789/0 conn(0x7f401c15fc50 legacy=0x7f401c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 -- 172.21.15.31:0/2962617155 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c0ce0b0 con 0x7f4008006150 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f401a7fc640 1 -- 172.21.15.31:0/2962617155 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0722c0 con 0x7f40080036b0 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.548+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-19T04:54:52.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.749+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.749+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.749+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.749+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:52.749+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-19T04:54:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.150+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.150+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.150+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.150+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.150+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-19T04:54:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.952+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.952+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-19T04:54:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.952+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-19T04:54:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.952+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-19T04:54:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:53.952+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-19T04:54:54.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377058994 0 0) 0x7f401c0ce0b0 con 0x7f4008006150 2024-01-19T04:54:54.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4004003520 con 0x7f4008006150 2024-01-19T04:54:54.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032692349 0 0) 0x7f4004003520 con 0x7f4008006150 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 msgr2=0x7f4008005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40080036b0 0x7f4008005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400800af50 con 0x7f4008006150 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f401000c570 con 0x7f4008006150 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 >> v1:172.21.15.31:6789/0 conn(0x7f4008006150 legacy=0x7f4008008610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f4021451640 1 -- 172.21.15.31:0/2962617155 reap_dead start 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f4004003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f4004004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f400400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0722c0 con 0x7f4004003970 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400400c1a0 con 0x7f40040047c0 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.754+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400400c360 con 0x7f4004005250 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f4004003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:54.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f4021451640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f400400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f401bfff640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f4004004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 msgr2=0x7f400400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f400400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 msgr2=0x7f4004004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f4004004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.755+0000 7f4020c50640 1 -- 172.21.15.31:0/2962617155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004003520 con 0x7f4004003970 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.756+0000 7f4020c50640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f4004003d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f401401a440 tx=0x7f40040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf86d14b7a1587af server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.756+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401402e960 con 0x7f4004003970 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.756+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4014032070 con 0x7f4004003970 2024-01-19T04:54:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.756+0000 7f401b7fe640 1 -- 172.21.15.31:0/2962617155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401402ed20 con 0x7f4004003970 2024-01-19T04:54:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 -- 172.21.15.31:0/2962617155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 msgr2=0x7f4004003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f4004003d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f401401a440 tx=0x7f40040057f0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 -- 172.21.15.31:0/2962617155 shutdown_connections 2024-01-19T04:54:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f400400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f4004004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 --2- 172.21.15.31:0/2962617155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f4004003d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 -- 172.21.15.31:0/2962617155 >> 172.21.15.31:0/2962617155 conn(0x7f401c0b1cc0 msgr2=0x7f401c0b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 -- 172.21.15.31:0/2962617155 shutdown_connections 2024-01-19T04:54:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.757+0000 7f4022eb4640 1 -- 172.21.15.31:0/2962617155 wait complete. 2024-01-19T04:54:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.758+0000 7f4022eb4640 1 Processor -- start 2024-01-19T04:54:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.758+0000 7f4022eb4640 1 -- start start 2024-01-19T04:54:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.759+0000 7f4022eb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f401c0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.759+0000 7f4022eb4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f401c0b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.759+0000 7f4022eb4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f401c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.759+0000 7f4022eb4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0caea0 con 0x7f4004003970 2024-01-19T04:54:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.759+0000 7f4022eb4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0cb180 con 0x7f40040047c0 2024-01-19T04:54:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.759+0000 7f4022eb4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0cb460 con 0x7f4004005250 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.759+0000 7f4020c50640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f401c0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f401c0ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44574/0 (socket says 172.21.15.31:44574) 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 -- 172.21.15.31:0/1310454364 learned_addr learned my addr 172.21.15.31:0/1310454364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4021451640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f401c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f401bfff640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f401c0b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 -- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 msgr2=0x7f401c15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f401c15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 -- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 msgr2=0x7f401c0b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f401c0b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 -- 172.21.15.31:0/1310454364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0cb740 con 0x7f4004003970 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4020c50640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f401c0ca810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f401403a040 tx=0x7f4014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f40197fa640 1 -- 172.21.15.31:0/1310454364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014038c40 con 0x7f4004003970 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f40197fa640 1 -- 172.21.15.31:0/1310454364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401400f040 con 0x7f4004003970 2024-01-19T04:54:54.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c0cba20 con 0x7f4004003970 2024-01-19T04:54:54.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f40197fa640 1 -- 172.21.15.31:0/1310454364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401402ee30 con 0x7f4004003970 2024-01-19T04:54:54.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.760+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c15af40 con 0x7f4004003970 2024-01-19T04:54:54.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.761+0000 7f40197fa640 1 -- 172.21.15.31:0/1310454364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f401404c450 con 0x7f4004003970 2024-01-19T04:54:54.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.761+0000 7f40197fa640 1 -- 172.21.15.31:0/1310454364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f40140435d0 con 0x7f4004003970 2024-01-19T04:54:54.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.762+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c074900 con 0x7f4004003970 2024-01-19T04:54:54.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.764+0000 7f40197fa640 1 -- 172.21.15.31:0/1310454364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f4014051060 con 0x7f4004003970 2024-01-19T04:54:54.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.832+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f401c06ea30 con 0x7f4004003970 2024-01-19T04:54:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.833+0000 7f40197fa640 1 -- 172.21.15.31:0/1310454364 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f401405b090 con 0x7f4004003970 2024-01-19T04:54:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 msgr2=0x7f401c0ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f401c0ca810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f401403a040 tx=0x7f4014004a40 comp rx=0 tx=0).stop 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 shutdown_connections 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4004005250 0x7f401c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40040047c0 0x7f401c0b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 --2- 172.21.15.31:0/1310454364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4004003970 0x7f401c0ca810 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 >> 172.21.15.31:0/1310454364 conn(0x7f401c0b1cc0 msgr2=0x7f401c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 shutdown_connections 2024-01-19T04:54:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:54.834+0000 7f4022eb4640 1 -- 172.21.15.31:0/1310454364 wait complete. 2024-01-19T04:54:54.845 INFO:tasks.ceph:check_enable_crimson: False 2024-01-19T04:54:54.845 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-19T04:54:54.845 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:54.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-19T04:54:54.872 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:54.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-19T04:54:54.940 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:54.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-19T04:54:55.006 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:54:55.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-19T04:54:55.073 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:55.073 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-19T04:54:55.106 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:55.106 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-19T04:54:55.176 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:55.176 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-19T04:54:55.246 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:54:55.246 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-19T04:54:55.316 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 51688160-057c-4b5a-951d-0a429534866c 0 2024-01-19T04:54:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.459+0000 7f56d5c5c640 1 Processor -- start 2024-01-19T04:54:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.459+0000 7f56d5c5c640 1 -- start start 2024-01-19T04:54:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56d5c5c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f56d00d3f10 0x7f56d00d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56d5c5c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56d00d1e40 0x7f56d00cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56d5c5c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d00d2290 con 0x7f56d015d5c0 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56d5c5c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d00d1450 con 0x7f56d00d1e40 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56d5c5c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d00cf240 con 0x7f56d00d3f10 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56ceffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f56d00d3f10 0x7f56d00d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56ceffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f56d00d3f10 0x7f56d00d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43550/0 (socket says 172.21.15.137:43550) 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.460+0000 7f56ceffd640 1 -- 172.21.15.137:0/3636981460 learned_addr learned my addr 172.21.15.137:0/3636981460 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.461+0000 7f56cffff640 1 --2- 172.21.15.137:0/3636981460 >> v2:172.21.15.31:3300/0 conn(0x7f56d00d1e40 0x7f56d00cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.461+0000 7f56ceffd640 1 -- 172.21.15.137:0/3636981460 >> v2:172.21.15.31:3300/0 conn(0x7f56d00d1e40 msgr2=0x7f56d00cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.461+0000 7f56ceffd640 1 --2- 172.21.15.137:0/3636981460 >> v2:172.21.15.31:3300/0 conn(0x7f56d00d1e40 0x7f56d00cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.461+0000 7f56ceffd640 1 -- 172.21.15.137:0/3636981460 >> v1:172.21.15.31:6789/0 conn(0x7f56d015d5c0 legacy=0x7f56d00cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.461+0000 7f56ceffd640 1 -- 172.21.15.137:0/3636981460 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d00ce6c0 con 0x7f56d00d3f10 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.461+0000 7f56ceffd640 1 --2- 172.21.15.137:0/3636981460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d00d1900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56d008c3b0 tx=0x7f56c402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74595cecdc63e319 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56ce7fc640 1 -- 172.21.15.137:0/3636981460 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4030860 con 0x7f56d00d3f10 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56ce7fc640 1 -- 172.21.15.137:0/3636981460 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c40309c0 con 0x7f56d00d3f10 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56ce7fc640 1 -- 172.21.15.137:0/3636981460 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4030d50 con 0x7f56d00d3f10 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 -- 172.21.15.137:0/3636981460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 msgr2=0x7f56d00d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 --2- 172.21.15.137:0/3636981460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d00d1900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56d008c3b0 tx=0x7f56c402b9a0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 -- 172.21.15.137:0/3636981460 shutdown_connections 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 --2- 172.21.15.137:0/3636981460 >> v2:172.21.15.31:3300/0 conn(0x7f56d00d1e40 0x7f56d00cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 --2- 172.21.15.137:0/3636981460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d00d1900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 -- 172.21.15.137:0/3636981460 >> 172.21.15.137:0/3636981460 conn(0x7f56d009c600 msgr2=0x7f56d00d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:55.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 -- 172.21.15.137:0/3636981460 shutdown_connections 2024-01-19T04:54:55.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.462+0000 7f56d5c5c640 1 -- 172.21.15.137:0/3636981460 wait complete. 2024-01-19T04:54:55.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.463+0000 7f56d5c5c640 1 Processor -- start 2024-01-19T04:54:55.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56d5c5c640 1 -- start start 2024-01-19T04:54:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56d5c5c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56d00d1e40 0x7f56d015bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56d5c5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56d5c5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015d5c0 0x7f56d00b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56d5c5c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d015f240 con 0x7f56d015d5c0 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56d5c5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d015e630 con 0x7f56d00d3f10 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56d5c5c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d015e910 con 0x7f56d00d1e40 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56ceffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56ceffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43552/0 (socket says 172.21.15.137:43552) 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56ceffd640 1 -- 172.21.15.137:0/1837858975 learned_addr learned my addr 172.21.15.137:0/1837858975 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56cf7fe640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56d00d1e40 0x7f56d015bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.464+0000 7f56cffff640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015d5c0 0x7f56d00b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56ceffd640 1 -- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56d00d1e40 msgr2=0x7f56d015bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56ceffd640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56d00d1e40 0x7f56d015bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56ceffd640 1 -- 172.21.15.137:0/1837858975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015d5c0 msgr2=0x7f56d00b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56ceffd640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015d5c0 0x7f56d00b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56ceffd640 1 -- 172.21.15.137:0/1837858975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40094d0 con 0x7f56d00d3f10 2024-01-19T04:54:55.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56ceffd640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d015dd30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f56c402b9e0 tx=0x7f56c4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56b7fff640 1 -- 172.21.15.137:0/1837858975 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4039d60 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56b7fff640 1 -- 172.21.15.137:0/1837858975 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4043400 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d015ebf0 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56b7fff640 1 -- 172.21.15.137:0/1837858975 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40437b0 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d015e270 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56b7fff640 1 -- 172.21.15.137:0/1837858975 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f56c4043c90 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.465+0000 7f56b7fff640 1 -- 172.21.15.137:0/1837858975 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f56c403e070 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.466+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d015f3b0 con 0x7f56d00d3f10 2024-01-19T04:54:55.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.468+0000 7f56b7fff640 1 -- 172.21.15.137:0/1837858975 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f56c4034070 con 0x7f56d00d3f10 2024-01-19T04:54:55.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.539+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "51688160-057c-4b5a-951d-0a429534866c", "id": 0} v 0) v1 -- 0x7f56d0074900 con 0x7f56d00d3f10 2024-01-19T04:54:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56b7fff640 1 -- 172.21.15.137:0/1837858975 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "51688160-057c-4b5a-951d-0a429534866c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f56c404d460 con 0x7f56d00d3f10 2024-01-19T04:54:55.542 INFO:teuthology.orchestra.run.smithi137.stdout:0 2024-01-19T04:54:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 msgr2=0x7f56d015dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56d5c5c640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d015dd30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f56c402b9e0 tx=0x7f56c4004730 comp rx=0 tx=0).stop 2024-01-19T04:54:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 shutdown_connections 2024-01-19T04:54:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56d5c5c640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015d5c0 0x7f56d00b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56d5c5c640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d00d3f10 0x7f56d015dd30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56d5c5c640 1 --2- 172.21.15.137:0/1837858975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56d00d1e40 0x7f56d015bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.543+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 >> 172.21.15.137:0/1837858975 conn(0x7f56d009c600 msgr2=0x7f56d006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:55.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.544+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 shutdown_connections 2024-01-19T04:54:55.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.544+0000 7f56d5c5c640 1 -- 172.21.15.137:0/1837858975 wait complete. 2024-01-19T04:54:55.553 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new c6763352-96f2-4658-b9b5-8e8c71d2a2a8 1 2024-01-19T04:54:55.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.654+0000 7f1d509b6640 1 Processor -- start 2024-01-19T04:54:55.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.654+0000 7f1d509b6640 1 -- start start 2024-01-19T04:54:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d509b6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d509b6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1d4c161e00 0x7f1d4c161260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d509b6640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c0cb7d0 con 0x7f1d4c1617a0 2024-01-19T04:54:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d509b6640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c161c30 con 0x7f1d4c161e00 2024-01-19T04:54:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d509b6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0cbab0 con 0x7f1d4c0b2370 2024-01-19T04:54:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d4a59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d4a59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:53696/0 (socket says 172.21.15.137:53696) 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d4a59c640 1 -- 172.21.15.137:0/233722536 learned_addr learned my addr 172.21.15.137:0/233722536 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d49d9b640 1 --2- 172.21.15.137:0/233722536 >> v2:172.21.15.31:3300/0 conn(0x7f1d4c161e00 0x7f1d4c161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d4a59c640 1 -- 172.21.15.137:0/233722536 >> v2:172.21.15.31:3300/0 conn(0x7f1d4c161e00 msgr2=0x7f1d4c161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d4a59c640 1 --2- 172.21.15.137:0/233722536 >> v2:172.21.15.31:3300/0 conn(0x7f1d4c161e00 0x7f1d4c161260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d4a59c640 1 -- 172.21.15.137:0/233722536 >> v1:172.21.15.31:6789/0 conn(0x7f1d4c1617a0 legacy=0x7f1d4c06b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.655+0000 7f1d4a59c640 1 -- 172.21.15.137:0/233722536 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c0cbd90 con 0x7f1d4c0b2370 2024-01-19T04:54:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.656+0000 7f1d4a59c640 1 --2- 172.21.15.137:0/233722536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0cffb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009850 tx=0x7f1d4402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84cfdd78c2a6736 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:55.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.656+0000 7f1d4959a640 1 -- 172.21.15.137:0/233722536 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44004230 con 0x7f1d4c0b2370 2024-01-19T04:54:55.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.656+0000 7f1d4959a640 1 -- 172.21.15.137:0/233722536 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44004390 con 0x7f1d4c0b2370 2024-01-19T04:54:55.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.656+0000 7f1d4959a640 1 -- 172.21.15.137:0/233722536 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44003810 con 0x7f1d4c0b2370 2024-01-19T04:54:55.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 -- 172.21.15.137:0/233722536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 msgr2=0x7f1d4c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 --2- 172.21.15.137:0/233722536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0cffb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009850 tx=0x7f1d4402b960 comp rx=0 tx=0).stop 2024-01-19T04:54:55.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 -- 172.21.15.137:0/233722536 shutdown_connections 2024-01-19T04:54:55.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 --2- 172.21.15.137:0/233722536 >> v2:172.21.15.31:3300/0 conn(0x7f1d4c161e00 0x7f1d4c161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 --2- 172.21.15.137:0/233722536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0cffb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 -- 172.21.15.137:0/233722536 >> 172.21.15.137:0/233722536 conn(0x7f1d4c08afe0 msgr2=0x7f1d4c0d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:55.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 -- 172.21.15.137:0/233722536 shutdown_connections 2024-01-19T04:54:55.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.657+0000 7f1d509b6640 1 -- 172.21.15.137:0/233722536 wait complete. 2024-01-19T04:54:55.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.658+0000 7f1d509b6640 1 Processor -- start 2024-01-19T04:54:55.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.658+0000 7f1d509b6640 1 -- start start 2024-01-19T04:54:55.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d509b6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d509b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d4c1617a0 0x7f1d4c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d509b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d4c161e00 0x7f1d4c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d509b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c14e340 con 0x7f1d4c1617a0 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d509b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c14e0b0 con 0x7f1d4c161e00 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d509b6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c14e890 con 0x7f1d4c0b2370 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d4a59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d4a59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:53708/0 (socket says 172.21.15.137:53708) 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d4a59c640 1 -- 172.21.15.137:0/3832056627 learned_addr learned my addr 172.21.15.137:0/3832056627 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d4ad9d640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d4c161e00 0x7f1d4c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.659+0000 7f1d49d9b640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d4c1617a0 0x7f1d4c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d4a59c640 1 -- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d4c161e00 msgr2=0x7f1d4c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d4a59c640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d4c161e00 0x7f1d4c0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d4a59c640 1 -- 172.21.15.137:0/3832056627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d4c1617a0 msgr2=0x7f1d4c0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d4a59c640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d4c1617a0 0x7f1d4c0b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d4a59c640 1 -- 172.21.15.137:0/3832056627 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009460 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d4ad9d640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d4c161e00 0x7f1d4c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d49d9b640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d4c1617a0 0x7f1d4c0b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d4a59c640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0b1790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009970 tx=0x7f1d440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d3affd640 1 -- 172.21.15.137:0/3832056627 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4403d070 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d3affd640 1 -- 172.21.15.137:0/3832056627 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44004500 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c0b5930 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d3affd640 1 -- 172.21.15.137:0/3832056627 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44003df0 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c14e520 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d3affd640 1 -- 172.21.15.137:0/3832056627 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f1d44006040 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.660+0000 7f1d3affd640 1 -- 172.21.15.137:0/3832056627 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f1d44042b90 con 0x7f1d4c0b2370 2024-01-19T04:54:55.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.661+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c14e6e0 con 0x7f1d4c0b2370 2024-01-19T04:54:55.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.663+0000 7f1d3affd640 1 -- 172.21.15.137:0/3832056627 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f1d44047070 con 0x7f1d4c0b2370 2024-01-19T04:54:55.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.737+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c6763352-96f2-4658-b9b5-8e8c71d2a2a8", "id": 1} v 0) v1 -- 0x7f1d4c0b5e50 con 0x7f1d4c0b2370 2024-01-19T04:54:55.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.742+0000 7f1d3affd640 1 -- 172.21.15.137:0/3832056627 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6763352-96f2-4658-b9b5-8e8c71d2a2a8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1d4c0b5e50 con 0x7f1d4c0b2370 2024-01-19T04:54:55.742 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 msgr2=0x7f1d4c0b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0b1790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009970 tx=0x7f1d440046c0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 shutdown_connections 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d4c161e00 0x7f1d4c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d4c1617a0 0x7f1d4c0b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 --2- 172.21.15.137:0/3832056627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d4c0b2370 0x7f1d4c0b1790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 >> 172.21.15.137:0/3832056627 conn(0x7f1d4c08afe0 msgr2=0x7f1d4c0ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 shutdown_connections 2024-01-19T04:54:55.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.743+0000 7f1d509b6640 1 -- 172.21.15.137:0/3832056627 wait complete. 2024-01-19T04:54:55.754 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 2e453883-d08d-4e0a-8e0e-cc6794eca4ff 2 2024-01-19T04:54:55.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.853+0000 7fa3f5f97640 1 Processor -- start 2024-01-19T04:54:55.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.853+0000 7fa3f5f97640 1 -- start start 2024-01-19T04:54:55.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3f5f97640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa3f006f900 0x7fa3f0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3f5f97640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f006a5b0 0x7fa3f00cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3f5f97640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa3f00cc1e0 0x7fa3f01610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3f5f97640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f006fce0 con 0x7fa3f00cc1e0 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3f5f97640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f015d9d0 con 0x7fa3f006f900 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3f5f97640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f006aa40 con 0x7fa3f006a5b0 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3ef7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa3f006f900 0x7fa3f0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3eeffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f006a5b0 0x7fa3f00cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3ef7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa3f006f900 0x7fa3f0071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43570/0 (socket says 172.21.15.137:43570) 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3ef7fe640 1 -- 172.21.15.137:0/1122801202 learned_addr learned my addr 172.21.15.137:0/1122801202 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3effff640 1 --2- 172.21.15.137:0/1122801202 >> v2:172.21.15.31:3300/0 conn(0x7fa3f00cc1e0 0x7fa3f01610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3eeffd640 1 -- 172.21.15.137:0/1122801202 >> v2:172.21.15.137:3300/0 conn(0x7fa3f006f900 msgr2=0x7fa3f0071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3eeffd640 1 --2- 172.21.15.137:0/1122801202 >> v2:172.21.15.137:3300/0 conn(0x7fa3f006f900 0x7fa3f0071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3eeffd640 1 -- 172.21.15.137:0/1122801202 >> v2:172.21.15.31:3300/0 conn(0x7fa3f00cc1e0 msgr2=0x7fa3f01610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3eeffd640 1 --2- 172.21.15.137:0/1122801202 >> v2:172.21.15.31:3300/0 conn(0x7fa3f00cc1e0 0x7fa3f01610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3eeffd640 1 -- 172.21.15.137:0/1122801202 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f006ad20 con 0x7fa3f006a5b0 2024-01-19T04:54:55.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.854+0000 7fa3ef7fe640 1 --2- 172.21.15.137:0/1122801202 >> v2:172.21.15.137:3300/0 conn(0x7fa3f006f900 0x7fa3f0071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:55.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.855+0000 7fa3eeffd640 1 --2- 172.21.15.137:0/1122801202 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f006a5b0 0x7fa3f00cbca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0074250 tx=0x7fa3d802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa29eb99a3b0629b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:55.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.855+0000 7fa3ee7fc640 1 -- 172.21.15.137:0/1122801202 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d802e070 con 0x7fa3f006a5b0 2024-01-19T04:54:55.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.855+0000 7fa3ee7fc640 1 -- 172.21.15.137:0/1122801202 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d80315a0 con 0x7fa3f006a5b0 2024-01-19T04:54:55.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.855+0000 7fa3ee7fc640 1 -- 172.21.15.137:0/1122801202 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8031930 con 0x7fa3f006a5b0 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 -- 172.21.15.137:0/1122801202 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f006a5b0 msgr2=0x7fa3f00cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/1122801202 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f006a5b0 0x7fa3f00cbca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0074250 tx=0x7fa3d802b840 comp rx=0 tx=0).stop 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 -- 172.21.15.137:0/1122801202 shutdown_connections 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/1122801202 >> v2:172.21.15.31:3300/0 conn(0x7fa3f00cc1e0 0x7fa3f01610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/1122801202 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f006a5b0 0x7fa3f00cbca0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/1122801202 >> v2:172.21.15.137:3300/0 conn(0x7fa3f006f900 0x7fa3f0071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 -- 172.21.15.137:0/1122801202 >> 172.21.15.137:0/1122801202 conn(0x7fa3f00d0000 msgr2=0x7fa3f00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 -- 172.21.15.137:0/1122801202 shutdown_connections 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.856+0000 7fa3f5f97640 1 -- 172.21.15.137:0/1122801202 wait complete. 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.857+0000 7fa3f5f97640 1 Processor -- start 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.857+0000 7fa3f5f97640 1 -- start start 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3f5f97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f006f900 0x7fa3f015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3f5f97640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f00cc1e0 0x7fa3f015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3f5f97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f00a0e30 0x7fa3f00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:55.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3f5f97640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01f63d0 con 0x7fa3f00a0e30 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3f5f97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01f66b0 con 0x7fa3f006f900 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3f5f97640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01f6990 con 0x7fa3f00cc1e0 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f006f900 0x7fa3f015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3eeffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f00cc1e0 0x7fa3f015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f006f900 0x7fa3f015b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43572/0 (socket says 172.21.15.137:43572) 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 -- 172.21.15.137:0/949461298 learned_addr learned my addr 172.21.15.137:0/949461298 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3effff640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f00a0e30 0x7fa3f00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 -- 172.21.15.137:0/949461298 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f00cc1e0 msgr2=0x7fa3f015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f00cc1e0 0x7fa3f015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 -- 172.21.15.137:0/949461298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f00a0e30 msgr2=0x7fa3f00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f00a0e30 0x7fa3f00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 -- 172.21.15.137:0/949461298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00094d0 con 0x7fa3f006f900 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3ef7fe640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f006f900 0x7fa3f015b8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000e9d0 tx=0x7fa3e000ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:55.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3e7fff640 1 -- 172.21.15.137:0/949461298 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000ca60 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3e7fff640 1 -- 172.21.15.137:0/949461298 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e000cbc0 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3e7fff640 1 -- 172.21.15.137:0/949461298 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0013470 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.858+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d8009400 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.859+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f015e420 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.859+0000 7fa3e7fff640 1 -- 172.21.15.137:0/949461298 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fa3e0005990 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.859+0000 7fa3e7fff640 1 -- 172.21.15.137:0/949461298 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa3e00139f0 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.860+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f015e610 con 0x7fa3f006f900 2024-01-19T04:54:55.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.862+0000 7fa3e7fff640 1 -- 172.21.15.137:0/949461298 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fa3e0021060 con 0x7fa3f006f900 2024-01-19T04:54:55.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.934+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2e453883-d08d-4e0a-8e0e-cc6794eca4ff", "id": 2} v 0) v1 -- 0x7fa3f0074900 con 0x7fa3f006f900 2024-01-19T04:54:55.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.937+0000 7fa3e7fff640 1 -- 172.21.15.137:0/949461298 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e453883-d08d-4e0a-8e0e-cc6794eca4ff", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa3e0017070 con 0x7fa3f006f900 2024-01-19T04:54:55.936 INFO:teuthology.orchestra.run.smithi137.stdout:2 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f006f900 msgr2=0x7fa3f015b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f006f900 0x7fa3f015b8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000e9d0 tx=0x7fa3e000ee90 comp rx=0 tx=0).stop 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 shutdown_connections 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f00a0e30 0x7fa3f00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3f00cc1e0 0x7fa3f015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 --2- 172.21.15.137:0/949461298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f006f900 0x7fa3f015b8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 >> 172.21.15.137:0/949461298 conn(0x7fa3f00d0000 msgr2=0x7fa3f0160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 shutdown_connections 2024-01-19T04:54:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:55.938+0000 7fa3f5f97640 1 -- 172.21.15.137:0/949461298 wait complete. 2024-01-19T04:54:55.948 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new fc395e3e-0c30-4a22-8af8-8621c00739ab 3 2024-01-19T04:54:55.970 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:55.970+0000 7effe13a3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-19T04:54:55.984 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:55.985+0000 7f9f84814040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-19T04:54:56.025 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.025+0000 7effe13a3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-19T04:54:56.040 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.040+0000 7f9f84814040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-19T04:54:56.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f98ee640 1 Processor -- start 2024-01-19T04:54:56.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f98ee640 1 -- start start 2024-01-19T04:54:56.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f98ee640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb5f415d720 0x7fb5f40d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f98ee640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5f40d3ea0 0x7fb5f40d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f98ee640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40cf160 con 0x7fb5f40d1a50 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f98ee640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f40d1ec0 con 0x7fb5f40d3ea0 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f98ee640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f40ce8c0 con 0x7fb5f415d720 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f2ffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb5f415d720 0x7fb5f40d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f2ffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb5f415d720 0x7fb5f40d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43580/0 (socket says 172.21.15.137:43580) 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.049+0000 7fb5f2ffd640 1 -- 172.21.15.137:0/231158536 learned_addr learned my addr 172.21.15.137:0/231158536 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.050+0000 7fb5f27fc640 1 --2- 172.21.15.137:0/231158536 >> v2:172.21.15.31:3300/0 conn(0x7fb5f40d3ea0 0x7fb5f40d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.050+0000 7fb5f2ffd640 1 -- 172.21.15.137:0/231158536 >> v2:172.21.15.31:3300/0 conn(0x7fb5f40d3ea0 msgr2=0x7fb5f40d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.050+0000 7fb5f2ffd640 1 --2- 172.21.15.137:0/231158536 >> v2:172.21.15.31:3300/0 conn(0x7fb5f40d3ea0 0x7fb5f40d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.050+0000 7fb5f2ffd640 1 -- 172.21.15.137:0/231158536 >> v1:172.21.15.137:6789/0 conn(0x7fb5f40d1a50 legacy=0x7fb5f40cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.050+0000 7fb5f2ffd640 1 -- 172.21.15.137:0/231158536 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40ceba0 con 0x7fb5f415d720 2024-01-19T04:54:56.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.050+0000 7fb5f2ffd640 1 --2- 172.21.15.137:0/231158536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f415d720 0x7fb5f40d0ee0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40cf350 tx=0x7fb5e402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48c537e7f4ca474 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f1ffb640 1 -- 172.21.15.137:0/231158536 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40317e0 con 0x7fb5f415d720 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f1ffb640 1 -- 172.21.15.137:0/231158536 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4031940 con 0x7fb5f415d720 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f1ffb640 1 -- 172.21.15.137:0/231158536 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4031cd0 con 0x7fb5f415d720 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 -- 172.21.15.137:0/231158536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f415d720 msgr2=0x7fb5f40d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 --2- 172.21.15.137:0/231158536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f415d720 0x7fb5f40d0ee0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40cf350 tx=0x7fb5e402b880 comp rx=0 tx=0).stop 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 -- 172.21.15.137:0/231158536 shutdown_connections 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 --2- 172.21.15.137:0/231158536 >> v2:172.21.15.31:3300/0 conn(0x7fb5f40d3ea0 0x7fb5f40d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 --2- 172.21.15.137:0/231158536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f415d720 0x7fb5f40d0ee0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 -- 172.21.15.137:0/231158536 >> 172.21.15.137:0/231158536 conn(0x7fb5f40d3310 msgr2=0x7fb5f40d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 -- 172.21.15.137:0/231158536 shutdown_connections 2024-01-19T04:54:56.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.051+0000 7fb5f98ee640 1 -- 172.21.15.137:0/231158536 wait complete. 2024-01-19T04:54:56.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.052+0000 7fb5f98ee640 1 Processor -- start 2024-01-19T04:54:56.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f98ee640 1 -- start start 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f98ee640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb5f40d1a50 0x7fb5f40c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f98ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f40d3ea0 0x7fb5f40c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f98ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f415d720 0x7fb5f41f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f98ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40c80a0 con 0x7fb5f415d720 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f98ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f41f6120 con 0x7fb5f40d3ea0 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f98ee640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f41f7070 con 0x7fb5f40d1a50 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f27fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f40d3ea0 0x7fb5f40c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f2ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb5f40d1a50 0x7fb5f40c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f27fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f40d3ea0 0x7fb5f40c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43596/0 (socket says 172.21.15.137:43596) 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f2ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb5f40d1a50 0x7fb5f40c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:53742/0 (socket says 172.21.15.137:53742) 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f27fc640 1 -- 172.21.15.137:0/1060723097 learned_addr learned my addr 172.21.15.137:0/1060723097 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.053+0000 7fb5f37fe640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f415d720 0x7fb5f41f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f2ffd640 1 -- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f40d3ea0 msgr2=0x7fb5f40c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f2ffd640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f40d3ea0 0x7fb5f40c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f2ffd640 1 -- 172.21.15.137:0/1060723097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f415d720 msgr2=0x7fb5f41f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f2ffd640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f415d720 0x7fb5f41f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f2ffd640 1 -- 172.21.15.137:0/1060723097 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40094d0 con 0x7fb5f40d1a50 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f27fc640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f40d3ea0 0x7fb5f40c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f2ffd640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb5f40d1a50 0x7fb5f40c73e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4033000 tx=0x7fb5e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5dbfff640 1 -- 172.21.15.137:0/1060723097 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4032880 con 0x7fb5f40d1a50 2024-01-19T04:54:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5dbfff640 1 -- 172.21.15.137:0/1060723097 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4032a20 con 0x7fb5f40d1a50 2024-01-19T04:54:56.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f40c7ae0 con 0x7fb5f40d1a50 2024-01-19T04:54:56.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5dbfff640 1 -- 172.21.15.137:0/1060723097 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4032dd0 con 0x7fb5f40d1a50 2024-01-19T04:54:56.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f41f6330 con 0x7fb5f40d1a50 2024-01-19T04:54:56.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5dbfff640 1 -- 172.21.15.137:0/1060723097 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fb5e404e450 con 0x7fb5f40d1a50 2024-01-19T04:54:56.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.054+0000 7fb5dbfff640 1 -- 172.21.15.137:0/1060723097 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fb5e4045a60 con 0x7fb5f40d1a50 2024-01-19T04:54:56.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.055+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f4074900 con 0x7fb5f40d1a50 2024-01-19T04:54:56.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.057+0000 7fb5dbfff640 1 -- 172.21.15.137:0/1060723097 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fb5e402e070 con 0x7fb5f40d1a50 2024-01-19T04:54:56.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.129+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fc395e3e-0c30-4a22-8af8-8621c00739ab", "id": 3} v 0) v1 -- 0x7fb5f40d03c0 con 0x7fb5f40d1a50 2024-01-19T04:54:56.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.132+0000 7fb5dbfff640 1 -- 172.21.15.137:0/1060723097 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fc395e3e-0c30-4a22-8af8-8621c00739ab", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb5e4045d00 con 0x7fb5f40d1a50 2024-01-19T04:54:56.132 INFO:teuthology.orchestra.run.smithi137.stdout:3 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb5f40d1a50 msgr2=0x7fb5f40c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb5f40d1a50 0x7fb5f40c73e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4033000 tx=0x7fb5e40046c0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 shutdown_connections 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f415d720 0x7fb5f41f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f40d3ea0 0x7fb5f40c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 --2- 172.21.15.137:0/1060723097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb5f40d1a50 0x7fb5f40c73e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 >> 172.21.15.137:0/1060723097 conn(0x7fb5f40d3310 msgr2=0x7fb5f406c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 shutdown_connections 2024-01-19T04:54:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.133+0000 7fb5f98ee640 1 -- 172.21.15.137:0/1060723097 wait complete. 2024-01-19T04:54:56.143 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 60ac628d-021f-40e5-8a3b-25b6911fa5a7 4 2024-01-19T04:54:56.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f820a5f5640 1 Processor -- start 2024-01-19T04:54:56.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f820a5f5640 1 -- start start 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f820a5f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82040b3720 0x7f82040cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f820a5f5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040d04b0 con 0x7f82040ce1c0 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f820a5f5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040ce610 con 0x7f820415b110 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f820a5f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82040cb930 con 0x7f82040b3720 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f8203fff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f820415b110 0x7f820415f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:44564/0 (socket says 172.21.15.137:44564) 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.284+0000 7f8203fff640 1 -- 172.21.15.137:0/697361533 learned_addr learned my addr 172.21.15.137:0/697361533 (peer_addr_for_me v1:172.21.15.137:0/0) 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.285+0000 7f82037fe640 1 --2- 172.21.15.137:0/697361533 >> v2:172.21.15.31:3300/0 conn(0x7f82040b3720 0x7f82040cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.285+0000 7f8202ffd640 1 -- 172.21.15.137:0/697361533 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608116551 0 0) 0x7f82040ce610 con 0x7f820415b110 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.285+0000 7f8202ffd640 1 -- 172.21.15.137:0/697361533 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ec003520 con 0x7f820415b110 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.285+0000 7f82037fe640 1 -- 172.21.15.137:0/697361533 >> v1:172.21.15.137:6789/0 conn(0x7f820415b110 legacy=0x7f820415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.285+0000 7f82037fe640 1 -- 172.21.15.137:0/697361533 >> v1:172.21.15.31:6789/0 conn(0x7f82040ce1c0 legacy=0x7f82040d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.285+0000 7f8202ffd640 1 -- 172.21.15.137:0/697361533 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454827264 0 0) 0x7f82040d04b0 con 0x7f82040ce1c0 2024-01-19T04:54:56.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.285+0000 7f82037fe640 1 -- 172.21.15.137:0/697361533 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040b3c00 con 0x7f82040b3720 2024-01-19T04:54:56.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.286+0000 7f82037fe640 1 --2- 172.21.15.137:0/697361533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82040b3720 0x7f82040cdc80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f81f80097a0 tx=0x7f81f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e3f8715e0b6f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.286+0000 7f8202ffd640 1 -- 172.21.15.137:0/697361533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8030610 con 0x7f82040b3720 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.286+0000 7f8202ffd640 1 -- 172.21.15.137:0/697361533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8030770 con 0x7f82040b3720 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.286+0000 7f8202ffd640 1 -- 172.21.15.137:0/697361533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8030b00 con 0x7f82040b3720 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.286+0000 7f820a5f5640 1 -- 172.21.15.137:0/697361533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82040b3720 msgr2=0x7f82040cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.286+0000 7f820a5f5640 1 --2- 172.21.15.137:0/697361533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82040b3720 0x7f82040cdc80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f81f80097a0 tx=0x7f81f802b960 comp rx=0 tx=0).stop 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.286+0000 7f820a5f5640 1 -- 172.21.15.137:0/697361533 shutdown_connections 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.287+0000 7f820a5f5640 1 --2- 172.21.15.137:0/697361533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82040b3720 0x7f82040cdc80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.287+0000 7f820a5f5640 1 -- 172.21.15.137:0/697361533 >> 172.21.15.137:0/697361533 conn(0x7f820406e970 msgr2=0x7f820406ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.287+0000 7f820a5f5640 1 -- 172.21.15.137:0/697361533 shutdown_connections 2024-01-19T04:54:56.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.287+0000 7f820a5f5640 1 -- 172.21.15.137:0/697361533 wait complete. 2024-01-19T04:54:56.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.288+0000 7f820a5f5640 1 Processor -- start 2024-01-19T04:54:56.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.288+0000 7f820a5f5640 1 -- start start 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82040b3720 0x7f82040caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82040ce1c0 0x7f820415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820415b110 0x7f820415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8204156a40 con 0x7f820415b110 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8204155e30 con 0x7f82040b3720 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8204156110 con 0x7f82040ce1c0 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8203fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82040b3720 0x7f82040caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f82037fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82040ce1c0 0x7f820415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8203fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82040b3720 0x7f82040caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43602/0 (socket says 172.21.15.137:43602) 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8203fff640 1 -- 172.21.15.137:0/3476819293 learned_addr learned my addr 172.21.15.137:0/3476819293 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8208b92640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820415b110 0x7f820415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f82037fe640 1 -- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82040b3720 msgr2=0x7f82040caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f82037fe640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82040b3720 0x7f82040caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f82037fe640 1 -- 172.21.15.137:0/3476819293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820415b110 msgr2=0x7f820415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f82037fe640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820415b110 0x7f820415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f82037fe640 1 -- 172.21.15.137:0/3476819293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009400 con 0x7f82040ce1c0 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8203fff640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82040b3720 0x7f82040caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f82037fe640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82040ce1c0 0x7f820415d020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81f8033000 tx=0x7f81f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8200ff9640 1 -- 172.21.15.137:0/3476819293 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f803e070 con 0x7f82040ce1c0 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8200ff9640 1 -- 172.21.15.137:0/3476819293 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8030e30 con 0x7f82040ce1c0 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82041563f0 con 0x7f82040ce1c0 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f8200ff9640 1 -- 172.21.15.137:0/3476819293 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f80399a0 con 0x7f82040ce1c0 2024-01-19T04:54:56.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.289+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82040cb1e0 con 0x7f82040ce1c0 2024-01-19T04:54:56.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.290+0000 7f8200ff9640 1 -- 172.21.15.137:0/3476819293 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f81f8042070 con 0x7f82040ce1c0 2024-01-19T04:54:56.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.290+0000 7f8200ff9640 1 -- 172.21.15.137:0/3476819293 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f81f8044930 con 0x7f82040ce1c0 2024-01-19T04:54:56.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.291+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8204074900 con 0x7f82040ce1c0 2024-01-19T04:54:56.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.293+0000 7f8200ff9640 1 -- 172.21.15.137:0/3476819293 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f81f8049070 con 0x7f82040ce1c0 2024-01-19T04:54:56.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.364+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "60ac628d-021f-40e5-8a3b-25b6911fa5a7", "id": 4} v 0) v1 -- 0x7f82041584c0 con 0x7f82040ce1c0 2024-01-19T04:54:56.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.370+0000 7f8200ff9640 1 -- 172.21.15.137:0/3476819293 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "60ac628d-021f-40e5-8a3b-25b6911fa5a7", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f81f8034070 con 0x7f82040ce1c0 2024-01-19T04:54:56.369 INFO:teuthology.orchestra.run.smithi137.stdout:4 2024-01-19T04:54:56.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82040ce1c0 msgr2=0x7f820415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82040ce1c0 0x7f820415d020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81f8033000 tx=0x7f81f80046c0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 shutdown_connections 2024-01-19T04:54:56.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820415b110 0x7f820415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82040ce1c0 0x7f820415d020 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 --2- 172.21.15.137:0/3476819293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82040b3720 0x7f82040caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 >> 172.21.15.137:0/3476819293 conn(0x7f820406e970 msgr2=0x7f82040b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 shutdown_connections 2024-01-19T04:54:56.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.371+0000 7f820a5f5640 1 -- 172.21.15.137:0/3476819293 wait complete. 2024-01-19T04:54:56.380 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 3d85949f-f4e2-498a-ab2f-f7ba5c6b275a 5 2024-01-19T04:54:56.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.481+0000 7fa0bbc41640 1 Processor -- start 2024-01-19T04:54:56.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.481+0000 7fa0bbc41640 1 -- start start 2024-01-19T04:54:56.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0bbc41640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa0b40ce090 0x7fa0b40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0bbc41640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b415e3e0 0x7fa0b415e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0bbc41640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b40cc890 con 0x7fa0b40cdcb0 2024-01-19T04:54:56.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0bbc41640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b40b0cf0 con 0x7fa0b40ce090 2024-01-19T04:54:56.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0bbc41640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40ccae0 con 0x7fa0b415e3e0 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0b91dc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa0b40ce090 0x7fa0b40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0ba1de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b415e3e0 0x7fa0b415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0ba1de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b415e3e0 0x7fa0b415e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:57336/0 (socket says 172.21.15.137:57336) 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0ba1de640 1 -- 172.21.15.137:0/636710771 learned_addr learned my addr 172.21.15.137:0/636710771 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.482+0000 7fa0b91dc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa0b40ce090 0x7fa0b40cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36066/0 (socket says 172.21.15.137:36066) 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.483+0000 7fa0b91dc640 1 -- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b415e3e0 msgr2=0x7fa0b415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.483+0000 7fa0b91dc640 1 --2- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b415e3e0 0x7fa0b415e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.483+0000 7fa0b91dc640 1 -- 172.21.15.137:0/636710771 >> v1:172.21.15.31:6789/0 conn(0x7fa0b40cdcb0 legacy=0x7fa0b40d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.483+0000 7fa0b91dc640 1 -- 172.21.15.137:0/636710771 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40ccdc0 con 0x7fa0b40ce090 2024-01-19T04:54:56.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.483+0000 7fa0ba1de640 1 --2- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b415e3e0 0x7fa0b415e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:56.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.483+0000 7fa0b91dc640 1 --2- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40ce090 0x7fa0b40cecb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00097a0 tx=0x7fa0b002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c620624bda8eb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0b89db640 1 -- 172.21.15.137:0/636710771 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b002d040 con 0x7fa0b40ce090 2024-01-19T04:54:56.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0b89db640 1 -- 172.21.15.137:0/636710771 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0031810 con 0x7fa0b40ce090 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0b89db640 1 -- 172.21.15.137:0/636710771 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0031ba0 con 0x7fa0b40ce090 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 -- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40ce090 msgr2=0x7fa0b40cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 --2- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40ce090 0x7fa0b40cecb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00097a0 tx=0x7fa0b002b8b0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 -- 172.21.15.137:0/636710771 shutdown_connections 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 --2- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b415e3e0 0x7fa0b415e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 --2- 172.21.15.137:0/636710771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40ce090 0x7fa0b40cecb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 -- 172.21.15.137:0/636710771 >> 172.21.15.137:0/636710771 conn(0x7fa0b408afe0 msgr2=0x7fa0b40d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 -- 172.21.15.137:0/636710771 shutdown_connections 2024-01-19T04:54:56.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.484+0000 7fa0bbc41640 1 -- 172.21.15.137:0/636710771 wait complete. 2024-01-19T04:54:56.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.485+0000 7fa0bbc41640 1 Processor -- start 2024-01-19T04:54:56.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0bbc41640 1 -- start start 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0bbc41640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40cdcb0 0x7fa0b415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0bbc41640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b40ce090 0x7fa0b40af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0bbc41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0b415e3e0 0x7fa0b40afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0bbc41640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40c84e0 con 0x7fa0b415e3e0 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0bbc41640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b5360 con 0x7fa0b40cdcb0 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0bbc41640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b5640 con 0x7fa0b40ce090 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0b99dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40cdcb0 0x7fa0b415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0b91dc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b40ce090 0x7fa0b40af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0b99dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40cdcb0 0x7fa0b415a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36078/0 (socket says 172.21.15.137:36078) 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0b99dd640 1 -- 172.21.15.137:0/1525679322 learned_addr learned my addr 172.21.15.137:0/1525679322 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0b91dc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b40ce090 0x7fa0b40af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:57346/0 (socket says 172.21.15.137:57346) 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.486+0000 7fa0ba1de640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0b415e3e0 0x7fa0b40afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0b91dc640 1 -- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40cdcb0 msgr2=0x7fa0b415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0b91dc640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40cdcb0 0x7fa0b415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0b91dc640 1 -- 172.21.15.137:0/1525679322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0b415e3e0 msgr2=0x7fa0b40afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0b91dc640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0b415e3e0 0x7fa0b40afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0b91dc640 1 -- 172.21.15.137:0/1525679322 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac009b30 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0b99dd640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40cdcb0 0x7fa0b415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0b91dc640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b40ce090 0x7fa0b40af5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0033000 tx=0x7fa0b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0aa7fc640 1 -- 172.21.15.137:0/1525679322 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0006e10 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0aa7fc640 1 -- 172.21.15.137:0/1525679322 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b003ad60 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0aa7fc640 1 -- 172.21.15.137:0/1525679322 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0044660 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b0009400 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b40c8750 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0aa7fc640 1 -- 172.21.15.137:0/1525679322 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fa0b0044b40 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.487+0000 7fa0aa7fc640 1 -- 172.21.15.137:0/1525679322 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa0b004d9c0 con 0x7fa0b40ce090 2024-01-19T04:54:56.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.488+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b4074900 con 0x7fa0b40ce090 2024-01-19T04:54:56.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.490+0000 7fa0aa7fc640 1 -- 172.21.15.137:0/1525679322 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fa0b0035020 con 0x7fa0b40ce090 2024-01-19T04:54:56.553 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.553+0000 7effe13a3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-19T04:54:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.563+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3d85949f-f4e2-498a-ab2f-f7ba5c6b275a", "id": 5} v 0) v1 -- 0x7fa0b40cd680 con 0x7fa0b40ce090 2024-01-19T04:54:56.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.568+0000 7fa0aa7fc640 1 -- 172.21.15.137:0/1525679322 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3d85949f-f4e2-498a-ab2f-f7ba5c6b275a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa0b004d440 con 0x7fa0b40ce090 2024-01-19T04:54:56.567 INFO:teuthology.orchestra.run.smithi137.stdout:5 2024-01-19T04:54:56.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.568+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b40ce090 msgr2=0x7fa0b40af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.568+0000 7fa0bbc41640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b40ce090 0x7fa0b40af5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0033000 tx=0x7fa0b00046c0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.569+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 shutdown_connections 2024-01-19T04:54:56.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.569+0000 7fa0bbc41640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0b415e3e0 0x7fa0b40afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.569+0000 7fa0bbc41640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa0b40ce090 0x7fa0b40af5d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.569+0000 7fa0bbc41640 1 --2- 172.21.15.137:0/1525679322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b40cdcb0 0x7fa0b415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.569+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 >> 172.21.15.137:0/1525679322 conn(0x7fa0b408afe0 msgr2=0x7fa0b40b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.569+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 shutdown_connections 2024-01-19T04:54:56.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.569+0000 7fa0bbc41640 1 -- 172.21.15.137:0/1525679322 wait complete. 2024-01-19T04:54:56.578 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new f5b3d372-3610-4a0c-b90b-fbd5e205cea4 6 2024-01-19T04:54:56.585 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.586+0000 7f9f84814040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-19T04:54:56.607 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.608+0000 7effe13a3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-19T04:54:56.641 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.642+0000 7f9f84814040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-19T04:54:56.653 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.654+0000 7effe13a3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-19T04:54:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.679+0000 7fdfb5e15640 1 Processor -- start 2024-01-19T04:54:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfb5e15640 1 -- start start 2024-01-19T04:54:56.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfb5e15640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fdfb006a5b0 0x7fdfb00cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfb5e15640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb0072550 con 0x7fdfb00cc1e0 2024-01-19T04:54:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfb5e15640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb015d9d0 con 0x7fdfb006f900 2024-01-19T04:54:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfb5e15640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb006fce0 con 0x7fdfb006a5b0 2024-01-19T04:54:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfaeffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fdfb006a5b0 0x7fdfb00cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfaeffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fdfb006a5b0 0x7fdfb00cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36088/0 (socket says 172.21.15.137:36088) 2024-01-19T04:54:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.680+0000 7fdfaeffd640 1 -- 172.21.15.137:0/1785391429 learned_addr learned my addr 172.21.15.137:0/1785391429 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.681+0000 7fdfaeffd640 1 -- 172.21.15.137:0/1785391429 >> v1:172.21.15.137:6789/0 conn(0x7fdfb006f900 legacy=0x7fdfb0071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.681+0000 7fdfaeffd640 1 -- 172.21.15.137:0/1785391429 >> v1:172.21.15.31:6789/0 conn(0x7fdfb00cc1e0 legacy=0x7fdfb00ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.681+0000 7fdfaeffd640 1 -- 172.21.15.137:0/1785391429 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb006aa40 con 0x7fdfb006a5b0 2024-01-19T04:54:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.681+0000 7fdfaeffd640 1 --2- 172.21.15.137:0/1785391429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb006a5b0 0x7fdfb00cbca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009870 tx=0x7fdfa002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2680845ae4e13c73 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfae7fc640 1 -- 172.21.15.137:0/1785391429 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa002f840 con 0x7fdfb006a5b0 2024-01-19T04:54:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfae7fc640 1 -- 172.21.15.137:0/1785391429 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa002f9a0 con 0x7fdfb006a5b0 2024-01-19T04:54:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfae7fc640 1 -- 172.21.15.137:0/1785391429 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa002fd30 con 0x7fdfb006a5b0 2024-01-19T04:54:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfb5e15640 1 -- 172.21.15.137:0/1785391429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb006a5b0 msgr2=0x7fdfb00cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfb5e15640 1 --2- 172.21.15.137:0/1785391429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb006a5b0 0x7fdfb00cbca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009870 tx=0x7fdfa002ba20 comp rx=0 tx=0).stop 2024-01-19T04:54:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfb5e15640 1 -- 172.21.15.137:0/1785391429 shutdown_connections 2024-01-19T04:54:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfb5e15640 1 --2- 172.21.15.137:0/1785391429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb006a5b0 0x7fdfb00cbca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfb5e15640 1 -- 172.21.15.137:0/1785391429 >> 172.21.15.137:0/1785391429 conn(0x7fdfb00d0000 msgr2=0x7fdfb00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfb5e15640 1 -- 172.21.15.137:0/1785391429 shutdown_connections 2024-01-19T04:54:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.682+0000 7fdfb5e15640 1 -- 172.21.15.137:0/1785391429 wait complete. 2024-01-19T04:54:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 Processor -- start 2024-01-19T04:54:56.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 -- start start 2024-01-19T04:54:56.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdfb006f900 0x7fdfb00c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfb00cc1e0 0x7fdfb009ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb009f140 0x7fdfb00b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b2bf0 con 0x7fdfb00cc1e0 2024-01-19T04:54:56.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00c6130 con 0x7fdfb009f140 2024-01-19T04:54:56.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.684+0000 7fdfb5e15640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b2e40 con 0x7fdfb006f900 2024-01-19T04:54:56.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdfb006f900 0x7fdfb00c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb009f140 0x7fdfb00b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdfb006f900 0x7fdfb00c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:57358/0 (socket says 172.21.15.137:57358) 2024-01-19T04:54:56.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 -- 172.21.15.137:0/3713843958 learned_addr learned my addr 172.21.15.137:0/3713843958 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaeffd640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfb00cc1e0 0x7fdfb009ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 -- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb009f140 msgr2=0x7fdfb00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb009f140 0x7fdfb00b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 -- 172.21.15.137:0/3713843958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfb00cc1e0 msgr2=0x7fdfb009ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfb00cc1e0 0x7fdfb009ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 -- 172.21.15.137:0/3713843958 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00094d0 con 0x7fdfb006f900 2024-01-19T04:54:56.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaffff640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb009f140 0x7fdfb00b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:56.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfaf7fe640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdfb006f900 0x7fdfb00c5bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c450 tx=0x7fdfa400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdf97fff640 1 -- 172.21.15.137:0/3713843958 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40052d0 con 0x7fdfb006f900 2024-01-19T04:54:56.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdf97fff640 1 -- 172.21.15.137:0/3713843958 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4005430 con 0x7fdfb006f900 2024-01-19T04:54:56.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb00b3120 con 0x7fdfb006f900 2024-01-19T04:54:56.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdf97fff640 1 -- 172.21.15.137:0/3713843958 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4012870 con 0x7fdfb006f900 2024-01-19T04:54:56.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb00b36e0 con 0x7fdfb006f900 2024-01-19T04:54:56.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdf97fff640 1 -- 172.21.15.137:0/3713843958 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fdfa4012d50 con 0x7fdfb006f900 2024-01-19T04:54:56.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.685+0000 7fdf97fff640 1 -- 172.21.15.137:0/3713843958 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fdfa401bcc0 con 0x7fdfb006f900 2024-01-19T04:54:56.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.686+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb00b38d0 con 0x7fdfb006f900 2024-01-19T04:54:56.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.688+0000 7fdf97fff640 1 -- 172.21.15.137:0/3713843958 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fdfa401b730 con 0x7fdfb006f900 2024-01-19T04:54:56.695 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.690+0000 7f9f84814040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-19T04:54:56.709 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.710+0000 7effe13a3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-19T04:54:56.744 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.745+0000 7f9f84814040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-19T04:54:56.753 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.753+0000 7effe13a3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-19T04:54:56.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.762+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f5b3d372-3610-4a0c-b90b-fbd5e205cea4", "id": 6} v 0) v1 -- 0x7fdfb0074900 con 0x7fdfb006f900 2024-01-19T04:54:56.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.767+0000 7fdf97fff640 1 -- 172.21.15.137:0/3713843958 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f5b3d372-3610-4a0c-b90b-fbd5e205cea4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdfa402a0b0 con 0x7fdfb006f900 2024-01-19T04:54:56.766 INFO:teuthology.orchestra.run.smithi137.stdout:6 2024-01-19T04:54:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdfb006f900 msgr2=0x7fdfb00c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdfb006f900 0x7fdfb00c5bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c450 tx=0x7fdfa400c910 comp rx=0 tx=0).stop 2024-01-19T04:54:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 shutdown_connections 2024-01-19T04:54:56.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfb009f140 0x7fdfb00b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfb00cc1e0 0x7fdfb009ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 --2- 172.21.15.137:0/3713843958 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdfb006f900 0x7fdfb00c5bf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 >> 172.21.15.137:0/3713843958 conn(0x7fdfb00d0000 msgr2=0x7fdfb006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 shutdown_connections 2024-01-19T04:54:56.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.768+0000 7fdfb5e15640 1 -- 172.21.15.137:0/3713843958 wait complete. 2024-01-19T04:54:56.777 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new ce9ec718-829b-4a4e-af88-86ab19fc4e9b 7 2024-01-19T04:54:56.789 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.790+0000 7f9f84814040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-19T04:54:56.858 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.859+0000 7effe13a3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-19T04:54:56.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.890+0000 7f0a34d90640 1 Processor -- start 2024-01-19T04:54:56.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.890+0000 7f0a34d90640 1 -- start start 2024-01-19T04:54:56.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a34d90640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a300add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a34d90640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f0a300cfad0 0x7f0a300cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a34d90640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a300ccc70 0x7f0a300b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a34d90640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a300b0a70 con 0x7f0a300ccc70 2024-01-19T04:54:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a34d90640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a300b18d0 con 0x7f0a300cfad0 2024-01-19T04:54:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a34d90640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300b0cc0 con 0x7f0a300cf6f0 2024-01-19T04:54:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a300add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a300add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:57364/0 (socket says 172.21.15.137:57364) 2024-01-19T04:54:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 -- 172.21.15.137:0/2200491384 learned_addr learned my addr 172.21.15.137:0/2200491384 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2dd9b640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.137:3300/0 conn(0x7f0a300cfad0 0x7f0a300cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2ed9d640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.31:3300/0 conn(0x7f0a300ccc70 0x7f0a300b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 -- 172.21.15.137:0/2200491384 >> v2:172.21.15.137:3300/0 conn(0x7f0a300cfad0 msgr2=0x7f0a300cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:56.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.137:3300/0 conn(0x7f0a300cfad0 0x7f0a300cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 -- 172.21.15.137:0/2200491384 >> v2:172.21.15.31:3300/0 conn(0x7f0a300ccc70 msgr2=0x7f0a300b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:56.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.31:3300/0 conn(0x7f0a300ccc70 0x7f0a300b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2e59c640 1 -- 172.21.15.137:0/2200491384 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300b0fa0 con 0x7f0a300cf6f0 2024-01-19T04:54:56.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2dd9b640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.137:3300/0 conn(0x7f0a300cfad0 0x7f0a300cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:56.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.891+0000 7f0a2ed9d640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.31:3300/0 conn(0x7f0a300ccc70 0x7f0a300b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:54:56.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.892+0000 7f0a2e59c640 1 --2- 172.21.15.137:0/2200491384 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a300add00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006dca0 tx=0x7f0a2002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af669d76363fa737 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.892+0000 7f0a2d59a640 1 -- 172.21.15.137:0/2200491384 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20031870 con 0x7f0a300cf6f0 2024-01-19T04:54:56.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.892+0000 7f0a2d59a640 1 -- 172.21.15.137:0/2200491384 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a200319d0 con 0x7f0a300cf6f0 2024-01-19T04:54:56.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.892+0000 7f0a2d59a640 1 -- 172.21.15.137:0/2200491384 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20031d60 con 0x7f0a300cf6f0 2024-01-19T04:54:56.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 -- 172.21.15.137:0/2200491384 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 msgr2=0x7f0a300add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 --2- 172.21.15.137:0/2200491384 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a300add00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006dca0 tx=0x7f0a2002b910 comp rx=0 tx=0).stop 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 -- 172.21.15.137:0/2200491384 shutdown_connections 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.137:3300/0 conn(0x7f0a300cfad0 0x7f0a300cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 --2- 172.21.15.137:0/2200491384 >> v2:172.21.15.31:3300/0 conn(0x7f0a300ccc70 0x7f0a300b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 --2- 172.21.15.137:0/2200491384 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a300add00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 -- 172.21.15.137:0/2200491384 >> 172.21.15.137:0/2200491384 conn(0x7f0a3008afe0 msgr2=0x7f0a300d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 -- 172.21.15.137:0/2200491384 shutdown_connections 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.893+0000 7f0a34d90640 1 -- 172.21.15.137:0/2200491384 wait complete. 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.894+0000 7f0a34d90640 1 Processor -- start 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.894+0000 7f0a34d90640 1 -- start start 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a34d90640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a300ccc70 0x7f0a300ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a34d90640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a301f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a34d90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a300cfad0 0x7f0a301f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a34d90640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300c89c0 con 0x7f0a300cfad0 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a34d90640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a301548b0 con 0x7f0a300ccc70 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a34d90640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300c86e0 con 0x7f0a300cf6f0 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a300ccc70 0x7f0a300ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2dd9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a301f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2dd9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a301f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:57378/0 (socket says 172.21.15.137:57378) 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a300ccc70 0x7f0a300ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36108/0 (socket says 172.21.15.137:36108) 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2dd9b640 1 -- 172.21.15.137:0/309709038 learned_addr learned my addr 172.21.15.137:0/309709038 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2ed9d640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a300cfad0 0x7f0a301f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 -- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 msgr2=0x7f0a301f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a301f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 -- 172.21.15.137:0/309709038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a300cfad0 msgr2=0x7f0a301f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a300cfad0 0x7f0a301f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 -- 172.21.15.137:0/309709038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a24009400 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2dd9b640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a301f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a2e59c640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a300ccc70 0x7f0a300ca1c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a20033000 tx=0x7f0a200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a1effd640 1 -- 172.21.15.137:0/309709038 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2003ac60 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.895+0000 7f0a1effd640 1 -- 172.21.15.137:0/309709038 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2003ae00 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.896+0000 7f0a1effd640 1 -- 172.21.15.137:0/309709038 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20044630 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.896+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a200094d0 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.896+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a300c5640 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.896+0000 7f0a1effd640 1 -- 172.21.15.137:0/309709038 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f0a20044790 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.896+0000 7f0a1effd640 1 -- 172.21.15.137:0/309709038 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f0a2002d040 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.896+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a300c51c0 con 0x7f0a300ccc70 2024-01-19T04:54:56.903 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.898+0000 7f9f84814040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-19T04:54:56.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.900+0000 7f0a1effd640 1 -- 172.21.15.137:0/309709038 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f0a300c51c0 con 0x7f0a300ccc70 2024-01-19T04:54:56.931 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.931+0000 7effe13a3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-19T04:54:56.970 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:56.971+0000 7f9f84814040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-19T04:54:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.972+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ce9ec718-829b-4a4e-af88-86ab19fc4e9b", "id": 7} v 0) v1 -- 0x7f0a3015cad0 con 0x7f0a300ccc70 2024-01-19T04:54:56.978 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:56.979+0000 7effe13a3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-19T04:54:56.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.980+0000 7f0a1effd640 1 -- 172.21.15.137:0/309709038 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce9ec718-829b-4a4e-af88-86ab19fc4e9b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0a3015cad0 con 0x7f0a300ccc70 2024-01-19T04:54:56.980 INFO:teuthology.orchestra.run.smithi137.stdout:7 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a300ccc70 msgr2=0x7f0a300ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a300ccc70 0x7f0a300ca1c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a20033000 tx=0x7f0a200046c0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 shutdown_connections 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a300cfad0 0x7f0a301f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0a300cf6f0 0x7f0a301f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 --2- 172.21.15.137:0/309709038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a300ccc70 0x7f0a300ca1c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 >> 172.21.15.137:0/309709038 conn(0x7f0a3008afe0 msgr2=0x7f0a3006fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 shutdown_connections 2024-01-19T04:54:56.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:54:56.981+0000 7f0a34d90640 1 -- 172.21.15.137:0/309709038 wait complete. 2024-01-19T04:54:56.991 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-19T04:54:56.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-19T04:54:56.993 INFO:tasks.ceph.osd.0:Started 2024-01-19T04:54:56.993 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-19T04:54:56.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-19T04:54:56.995 INFO:tasks.ceph.osd.1:Started 2024-01-19T04:54:56.996 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-19T04:54:56.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-19T04:54:56.998 INFO:tasks.ceph.osd.2:Started 2024-01-19T04:54:56.998 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-19T04:54:56.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-19T04:54:57.000 INFO:tasks.ceph.osd.3:Started 2024-01-19T04:54:57.000 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-19T04:54:57.000 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-19T04:54:57.002 INFO:tasks.ceph.osd.4:Started 2024-01-19T04:54:57.002 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-19T04:54:57.002 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-19T04:54:57.004 INFO:tasks.ceph.osd.5:Started 2024-01-19T04:54:57.004 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-19T04:54:57.004 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-19T04:54:57.006 INFO:tasks.ceph.osd.6:Started 2024-01-19T04:54:57.006 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-19T04:54:57.006 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-19T04:54:57.008 INFO:tasks.ceph.osd.7:Started 2024-01-19T04:54:57.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:54:57.024 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.025+0000 7f9f84814040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-19T04:54:57.052 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:57.053+0000 7effe13a3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-19T04:54:57.084 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.085+0000 7f9f84814040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-19T04:54:57.117 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:57.118+0000 7effe13a3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-19T04:54:57.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.128+0000 7f43d933c640 1 Processor -- start 2024-01-19T04:54:57.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.128+0000 7f43d933c640 1 -- start start 2024-01-19T04:54:57.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d933c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d40d08a0 0x7f43d415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:57.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d933c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43d406ecb0 0x7f43d40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:57.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d933c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d40d0040 con 0x7f43d406e8d0 2024-01-19T04:54:57.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d933c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d415c800 con 0x7f43d406ecb0 2024-01-19T04:54:57.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d933c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40cd500 con 0x7f43d40d08a0 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d2ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f43d406e8d0 0x7f43d40cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46650/0 (socket says 172.21.15.31:46650) 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d2ffd640 1 -- 172.21.15.31:0/2132431161 learned_addr learned my addr 172.21.15.31:0/2132431161 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d37fe640 1 --2- 172.21.15.31:0/2132431161 >> v2:172.21.15.31:3300/0 conn(0x7f43d406ecb0 0x7f43d40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.129+0000 7f43d27fc640 1 --2- 172.21.15.31:0/2132431161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d40d08a0 0x7f43d415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.130+0000 7f43d1ffb640 1 -- 172.21.15.31:0/2132431161 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179520423 0 0) 0x7f43d40d0040 con 0x7f43d406e8d0 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.130+0000 7f43d1ffb640 1 -- 172.21.15.31:0/2132431161 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43bc003520 con 0x7f43d406e8d0 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.130+0000 7f43d27fc640 1 -- 172.21.15.31:0/2132431161 >> v2:172.21.15.31:3300/0 conn(0x7f43d406ecb0 msgr2=0x7f43d40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.130+0000 7f43d27fc640 1 --2- 172.21.15.31:0/2132431161 >> v2:172.21.15.31:3300/0 conn(0x7f43d406ecb0 0x7f43d40cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.130+0000 7f43d27fc640 1 -- 172.21.15.31:0/2132431161 >> v1:172.21.15.31:6789/0 conn(0x7f43d406e8d0 legacy=0x7f43d40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.130+0000 7f43d27fc640 1 -- 172.21.15.31:0/2132431161 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40cb800 con 0x7f43d40d08a0 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.130+0000 7f43d37fe640 1 --2- 172.21.15.31:0/2132431161 >> v2:172.21.15.31:3300/0 conn(0x7f43d406ecb0 0x7f43d40cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d27fc640 1 --2- 172.21.15.31:0/2132431161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d40d08a0 0x7f43d415c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f43c800dbc0 tx=0x7f43c802fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a493f5cf4993416 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:57.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d1ffb640 1 -- 172.21.15.31:0/2132431161 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80307d0 con 0x7f43d40d08a0 2024-01-19T04:54:57.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d1ffb640 1 -- 172.21.15.31:0/2132431161 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8030930 con 0x7f43d40d08a0 2024-01-19T04:54:57.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d1ffb640 1 -- 172.21.15.31:0/2132431161 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8030cc0 con 0x7f43d40d08a0 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 -- 172.21.15.31:0/2132431161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d40d08a0 msgr2=0x7f43d415c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 --2- 172.21.15.31:0/2132431161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d40d08a0 0x7f43d415c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f43c800dbc0 tx=0x7f43c802fc30 comp rx=0 tx=0).stop 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 -- 172.21.15.31:0/2132431161 shutdown_connections 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 --2- 172.21.15.31:0/2132431161 >> v2:172.21.15.31:3300/0 conn(0x7f43d406ecb0 0x7f43d40cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 --2- 172.21.15.31:0/2132431161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d40d08a0 0x7f43d415c2c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 -- 172.21.15.31:0/2132431161 >> 172.21.15.31:0/2132431161 conn(0x7f43d40b2c60 msgr2=0x7f43d40b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 -- 172.21.15.31:0/2132431161 shutdown_connections 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.131+0000 7f43d933c640 1 -- 172.21.15.31:0/2132431161 wait complete. 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.133+0000 7f43d933c640 1 Processor -- start 2024-01-19T04:54:57.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.133+0000 7f43d933c640 1 -- start start 2024-01-19T04:54:57.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d933c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d406e8d0 0x7f43d4155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d933c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d406ecb0 0x7f43d40c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d933c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43d40d08a0 0x7f43d4157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d933c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40c4de0 con 0x7f43d40d08a0 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d933c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40c5010 con 0x7f43d406ecb0 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d933c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4155dc0 con 0x7f43d406e8d0 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d406e8d0 0x7f43d4155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d406e8d0 0x7f43d4155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:58420/0 (socket says 172.21.15.31:58420) 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 -- 172.21.15.31:0/2050681817 learned_addr learned my addr 172.21.15.31:0/2050681817 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d27fc640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d406ecb0 0x7f43d40c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d37fe640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43d40d08a0 0x7f43d4157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 -- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d406ecb0 msgr2=0x7f43d40c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d406ecb0 0x7f43d40c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 -- 172.21.15.31:0/2050681817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43d40d08a0 msgr2=0x7f43d4157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43d40d08a0 0x7f43d4157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d2ffd640 1 -- 172.21.15.31:0/2050681817 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc009a70 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d27fc640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d406ecb0 0x7f43d40c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.134+0000 7f43d37fe640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43d40d08a0 0x7f43d4157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43d2ffd640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d406e8d0 0x7f43d4155880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c390 tx=0x7f43c400c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2050681817 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4005ce0 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2050681817 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4005e40 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c800d8a0 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2050681817 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40072f0 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d40c9900 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2050681817 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f43c401c440 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.135+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2050681817 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f43c4012e40 con 0x7f43d406e8d0 2024-01-19T04:54:57.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.136+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d4074a80 con 0x7f43d406e8d0 2024-01-19T04:54:57.138 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.136+0000 7f9f84814040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-19T04:54:57.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.138+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2050681817 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f43c4017070 con 0x7f43d406e8d0 2024-01-19T04:54:57.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.199+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43d40696e0 con 0x7f43d406e8d0 2024-01-19T04:54:57.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.200+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2050681817 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f43c4012770 con 0x7f43d406e8d0 2024-01-19T04:54:57.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:54:57.200 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:54:56.974222+0000","last_up_change":"0.000000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d406e8d0 msgr2=0x7f43d4155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d406e8d0 0x7f43d4155880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c390 tx=0x7f43c400c790 comp rx=0 tx=0).stop 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 shutdown_connections 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43d40d08a0 0x7f43d4157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d406ecb0 0x7f43d40c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 --2- 172.21.15.31:0/2050681817 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43d406e8d0 0x7f43d4155880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 >> 172.21.15.31:0/2050681817 conn(0x7f43d40b2c60 msgr2=0x7f43d4158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 shutdown_connections 2024-01-19T04:54:57.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:54:57.201+0000 7f43d933c640 1 -- 172.21.15.31:0/2050681817 wait complete. 2024-01-19T04:54:57.211 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-19T04:54:57.211 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-19T04:54:57.513 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.513+0000 7f9f84814040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-19T04:54:57.515 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:57.516+0000 7effe13a3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-19T04:54:57.604 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-01-19T04:54:57.604+0000 7fb6cfb174c0 -1 Falling back to public interface 2024-01-19T04:54:57.615 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-01-19T04:54:57.615+0000 7f111860e4c0 -1 Falling back to public interface 2024-01-19T04:54:57.615 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-01-19T04:54:57.615+0000 7f761b61d4c0 -1 Falling back to public interface 2024-01-19T04:54:57.618 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-01-19T04:54:57.619+0000 7f074b2144c0 -1 Falling back to public interface 2024-01-19T04:54:57.623 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-01-19T04:54:57.623+0000 7fcd318c74c0 -1 Falling back to public interface 2024-01-19T04:54:57.626 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-01-19T04:54:57.627+0000 7f9d46ecf4c0 -1 Falling back to public interface 2024-01-19T04:54:57.626 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-01-19T04:54:57.627+0000 7f5848f224c0 -1 Falling back to public interface 2024-01-19T04:54:57.626 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-01-19T04:54:57.627+0000 7f9d45e76640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-01-19T04:54:57.627 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-01-19T04:54:57.627+0000 7fe4cc4fa4c0 -1 Falling back to public interface 2024-01-19T04:54:57.627 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-01-19T04:54:57.628+0000 7f5847ec9640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6814/0: (98) Address already in use 2024-01-19T04:54:57.648 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.649+0000 7f9f84814040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-19T04:54:57.658 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:57.658+0000 7effe13a3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-19T04:54:57.767 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.768+0000 7f9f84814040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-19T04:54:57.778 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:57.778+0000 7effe13a3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-19T04:54:57.812 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.813+0000 7f9f84814040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-19T04:54:57.825 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:57.825+0000 7effe13a3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-19T04:54:57.909 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:57.910+0000 7f9f84814040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-19T04:54:57.926 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:57.926+0000 7effe13a3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-19T04:54:58.004 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:58.005+0000 7f9f84814040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-19T04:54:58.017 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-19T04:54:58.024 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:58.024+0000 7effe13a3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-19T04:54:58.106 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.106+0000 7f6fbb32d640 1 Processor -- start 2024-01-19T04:54:58.107 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.106+0000 7f6fbb32d640 1 -- start start 2024-01-19T04:54:58.108 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.107+0000 7f6fbb32d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb40cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:58.108 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.107+0000 7f6fbb32d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb4141f00 con 0x7f6fb40b0cc0 2024-01-19T04:54:58.108 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.107+0000 7f6fbb32d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb40b3390 con 0x7f6fb4141b20 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.107+0000 7f6fbb32d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40b3630 con 0x7f6fb40b1130 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.107+0000 7f6fb88c8640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6fb40b0cc0 0x7f6fb40ccf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46658/0 (socket says 172.21.15.31:46658) 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.107+0000 7f6fb88c8640 1 -- 172.21.15.31:0/2334180101 learned_addr learned my addr 172.21.15.31:0/2334180101 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.107+0000 7f6fb98ca640 1 --2- 172.21.15.31:0/2334180101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb40cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fabfff640 1 -- 172.21.15.31:0/2334180101 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13644992 0 0) 0x7f6fb4141f00 con 0x7f6fb40b0cc0 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fabfff640 1 -- 172.21.15.31:0/2334180101 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa0003520 con 0x7f6fb40b0cc0 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fabfff640 1 -- 172.21.15.31:0/2334180101 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273874451 0 0) 0x7f6fb40b3390 con 0x7f6fb4141b20 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fabfff640 1 -- 172.21.15.31:0/2334180101 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb4141f00 con 0x7f6fb4141b20 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fb98ca640 1 -- 172.21.15.31:0/2334180101 >> v1:172.21.15.137:6789/0 conn(0x7f6fb4141b20 legacy=0x7f6fb406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fb98ca640 1 -- 172.21.15.31:0/2334180101 >> v1:172.21.15.31:6789/0 conn(0x7f6fb40b0cc0 legacy=0x7f6fb40ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:58.109 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fb98ca640 1 -- 172.21.15.31:0/2334180101 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40b3910 con 0x7f6fb40b1130 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.108+0000 7f6fb98ca640 1 --2- 172.21.15.31:0/2334180101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb40cd620 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009dc0 tx=0x7f6fac02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3d82ba8fae9c92 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fabfff640 1 -- 172.21.15.31:0/2334180101 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac02cc40 con 0x7f6fb40b1130 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fabfff640 1 -- 172.21.15.31:0/2334180101 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac02cda0 con 0x7f6fb40b1130 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fabfff640 1 -- 172.21.15.31:0/2334180101 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac034660 con 0x7f6fb40b1130 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fbb32d640 1 -- 172.21.15.31:0/2334180101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 msgr2=0x7f6fb40cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fbb32d640 1 --2- 172.21.15.31:0/2334180101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb40cd620 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009dc0 tx=0x7f6fac02be90 comp rx=0 tx=0).stop 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fbb32d640 1 -- 172.21.15.31:0/2334180101 shutdown_connections 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fbb32d640 1 --2- 172.21.15.31:0/2334180101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb40cd620 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fbb32d640 1 -- 172.21.15.31:0/2334180101 >> 172.21.15.31:0/2334180101 conn(0x7f6fb406dee0 msgr2=0x7f6fb406a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fbb32d640 1 -- 172.21.15.31:0/2334180101 shutdown_connections 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.109+0000 7f6fbb32d640 1 -- 172.21.15.31:0/2334180101 wait complete. 2024-01-19T04:54:58.111 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 Processor -- start 2024-01-19T04:54:58.112 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 -- start start 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb40b0cc0 0x7f6fb4131420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb4141b20 0x7f6fb40ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4131960 con 0x7f6fb4141b20 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb415e140 con 0x7f6fb40b0cc0 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fbb32d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb415d530 con 0x7f6fb40b1130 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fb98ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb4141b20 0x7f6fb40ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:58.114 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fb98ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb4141b20 0x7f6fb40ab570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44664/0 (socket says 172.21.15.31:44664) 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fb98ca640 1 -- 172.21.15.31:0/3792734350 learned_addr learned my addr 172.21.15.31:0/3792734350 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fb90c9640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb40b0cc0 0x7f6fb4131420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.111+0000 7f6fb88c8640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fb98ca640 1 -- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 msgr2=0x7f6fb415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fb98ca640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb415cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fb98ca640 1 -- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb40b0cc0 msgr2=0x7f6fb4131420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fb98ca640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb40b0cc0 0x7f6fb4131420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fb98ca640 1 -- 172.21.15.31:0/3792734350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac009a70 con 0x7f6fb4141b20 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fb98ca640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb4141b20 0x7f6fb40ab570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6fac037040 tx=0x7f6fac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fa9ffb640 1 -- 172.21.15.31:0/3792734350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac040650 con 0x7f6fb4141b20 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fa9ffb640 1 -- 172.21.15.31:0/3792734350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac0407b0 con 0x7f6fb4141b20 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb415d810 con 0x7f6fb4141b20 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fa9ffb640 1 -- 172.21.15.31:0/3792734350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac040b60 con 0x7f6fb4141b20 2024-01-19T04:54:58.115 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb415ddd0 con 0x7f6fb4141b20 2024-01-19T04:54:58.116 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fa9ffb640 1 -- 172.21.15.31:0/3792734350 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f6fac003d10 con 0x7f6fb4141b20 2024-01-19T04:54:58.116 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.112+0000 7f6fa9ffb640 1 -- 172.21.15.31:0/3792734350 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6fac004700 con 0x7f6fb4141b20 2024-01-19T04:54:58.116 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.113+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4140aa0 con 0x7f6fb4141b20 2024-01-19T04:54:58.116 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.115+0000 7f6fa9ffb640 1 -- 172.21.15.31:0/3792734350 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f6fac044020 con 0x7f6fb4141b20 2024-01-19T04:54:58.143 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:58.143+0000 7f9f84814040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-19T04:54:58.173 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:58.173+0000 7effe13a3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-19T04:54:58.177 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.177+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fb415df60 con 0x7f6fb4141b20 2024-01-19T04:54:58.178 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.178+0000 7f6fa9ffb640 1 -- 172.21.15.31:0/3792734350 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f6fac039070 con 0x7f6fb4141b20 2024-01-19T04:54:58.178 INFO:teuthology.misc.health.smithi031.stdout: 2024-01-19T04:54:58.178 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:54:56.974222+0000","last_up_change":"0.000000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:54:58.179 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb4141b20 msgr2=0x7f6fb40ab570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:54:58.179 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb4141b20 0x7f6fb40ab570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6fac037040 tx=0x7f6fac004a40 comp rx=0 tx=0).stop 2024-01-19T04:54:58.179 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 shutdown_connections 2024-01-19T04:54:58.179 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb4141b20 0x7f6fb40ab570 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:58.180 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb40b1130 0x7f6fb415cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:58.180 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 --2- 172.21.15.31:0/3792734350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb40b0cc0 0x7f6fb4131420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:54:58.180 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 >> 172.21.15.31:0/3792734350 conn(0x7f6fb406dee0 msgr2=0x7f6fb4140dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:54:58.180 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 shutdown_connections 2024-01-19T04:54:58.180 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:54:58.179+0000 7f6fbb32d640 1 -- 172.21.15.31:0/3792734350 wait complete. 2024-01-19T04:54:58.185 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:58.186+0000 7f9f84814040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-19T04:54:58.188 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-19T04:54:58.218 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:58.218+0000 7effe13a3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-19T04:54:58.325 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:58.325+0000 7f9f84814040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-19T04:54:58.364 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:58.364+0000 7effe13a3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-19T04:54:59.114 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:59.114+0000 7f9f84814040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-19T04:54:59.209 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:59.209+0000 7effe13a3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-19T04:54:59.383 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:59.384+0000 7f9f84814040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-19T04:54:59.434 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:59.434+0000 7f9f84814040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-19T04:54:59.494 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:59.493+0000 7effe13a3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-19T04:54:59.533 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:54:59.534+0000 7f9f84814040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-19T04:54:59.547 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:59.547+0000 7effe13a3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-19T04:54:59.653 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-01-19T04:54:59.652+0000 7effe13a3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-19T04:55:00.561 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:55:00.561+0000 7f9f258ab640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-19T04:55:00.562 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-19T04:55:00.562+0000 7f9f258ab640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-19T04:55:01.078 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-01-19T04:55:01.078+0000 7fb6cfb174c0 -1 osd.1 0 log_to_monitors true 2024-01-19T04:55:01.118 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-01-19T04:55:01.118+0000 7f111860e4c0 -1 osd.0 0 log_to_monitors true 2024-01-19T04:55:01.150 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-01-19T04:55:01.150+0000 7fcd318c74c0 -1 osd.2 0 log_to_monitors true 2024-01-19T04:55:01.162 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-01-19T04:55:01.163+0000 7f074b2144c0 -1 osd.7 0 log_to_monitors true 2024-01-19T04:55:01.163 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-01-19T04:55:01.163+0000 7f761b61d4c0 -1 osd.3 0 log_to_monitors true 2024-01-19T04:55:01.165 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-01-19T04:55:01.166+0000 7f9d46ecf4c0 -1 osd.5 0 log_to_monitors true 2024-01-19T04:55:01.169 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-01-19T04:55:01.170+0000 7f5848f224c0 -1 osd.6 0 log_to_monitors true 2024-01-19T04:55:01.175 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-01-19T04:55:01.175+0000 7fe4cc4fa4c0 -1 osd.4 0 log_to_monitors true 2024-01-19T04:55:04.584 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-01-19T04:55:04.585+0000 7fb6cb282640 -1 osd.1 0 waiting for initial osdmap 2024-01-19T04:55:04.588 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-01-19T04:55:04.589+0000 7fb6c38bb640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.603 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-01-19T04:55:04.604+0000 7f1113d79640 -1 osd.0 0 waiting for initial osdmap 2024-01-19T04:55:04.615 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-01-19T04:55:04.616+0000 7f110c3b2640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.616 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-01-19T04:55:04.617+0000 7fe4c7c67640 -1 osd.4 0 waiting for initial osdmap 2024-01-19T04:55:04.628 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-01-19T04:55:04.628+0000 7f0746386640 -1 osd.7 0 waiting for initial osdmap 2024-01-19T04:55:04.628 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-01-19T04:55:04.629+0000 7fe4c02a0640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.639 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-01-19T04:55:04.640+0000 7f073e9bf640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.639 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-01-19T04:55:04.640+0000 7f9d4263a640 -1 osd.5 0 waiting for initial osdmap 2024-01-19T04:55:04.650 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-01-19T04:55:04.651+0000 7f9d3ac73640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.660 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-01-19T04:55:04.661+0000 7f584468d640 -1 osd.6 0 waiting for initial osdmap 2024-01-19T04:55:04.664 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-01-19T04:55:04.664+0000 7fcd2d032640 -1 osd.2 0 waiting for initial osdmap 2024-01-19T04:55:04.675 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-01-19T04:55:04.676+0000 7fcd2566b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.676 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-01-19T04:55:04.677+0000 7f583ccc6640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.677 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-01-19T04:55:04.678+0000 7f76167e9640 -1 osd.3 0 waiting for initial osdmap 2024-01-19T04:55:04.689 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-01-19T04:55:04.690+0000 7f760ee22640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-19T04:55:04.995 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:05.086 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.086+0000 7f3458a89640 1 Processor -- start 2024-01-19T04:55:05.086 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.086+0000 7f3458a89640 1 -- start start 2024-01-19T04:55:05.087 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f3458a89640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f345415d540 0x7f34540b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:05.087 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f3458a89640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d1400 0x7f34540d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:05.087 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f3458a89640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345415d920 con 0x7f34540d32e0 2024-01-19T04:55:05.087 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f3458a89640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34540d1970 con 0x7f345415d540 2024-01-19T04:55:05.087 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f3458a89640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34540d1c50 con 0x7f34540d1400 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f345259c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f345415d540 0x7f34540b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f3452d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f34540d32e0 0x7f34540b64f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39274/0 (socket says 172.21.15.31:39274) 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.087+0000 7f3452d9d640 1 -- 172.21.15.31:0/2925963722 learned_addr learned my addr 172.21.15.31:0/2925963722 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f344bfff640 1 --2- 172.21.15.31:0/2925963722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d1400 0x7f34540d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f345259c640 1 -- 172.21.15.31:0/2925963722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d1400 msgr2=0x7f34540d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f345259c640 1 --2- 172.21.15.31:0/2925963722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d1400 0x7f34540d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f345259c640 1 -- 172.21.15.31:0/2925963722 >> v1:172.21.15.31:6789/0 conn(0x7f34540d32e0 legacy=0x7f34540b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f3451d9b640 1 -- 172.21.15.31:0/2925963722 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584536136 0 0) 0x7f345415d920 con 0x7f34540d32e0 2024-01-19T04:55:05.088 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f345259c640 1 -- 172.21.15.31:0/2925963722 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34540d1f30 con 0x7f345415d540 2024-01-19T04:55:05.089 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f345259c640 1 --2- 172.21.15.31:0/2925963722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b8ce0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f34440097a0 tx=0x7f344402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b851321fd421a781 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:05.089 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.088+0000 7f3451d9b640 1 -- 172.21.15.31:0/2925963722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444031830 con 0x7f345415d540 2024-01-19T04:55:05.089 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3451d9b640 1 -- 172.21.15.31:0/2925963722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444031990 con 0x7f345415d540 2024-01-19T04:55:05.089 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3451d9b640 1 -- 172.21.15.31:0/2925963722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444031d20 con 0x7f345415d540 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 -- 172.21.15.31:0/2925963722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 msgr2=0x7f34540b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 --2- 172.21.15.31:0/2925963722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b8ce0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f34440097a0 tx=0x7f344402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 -- 172.21.15.31:0/2925963722 shutdown_connections 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 --2- 172.21.15.31:0/2925963722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d1400 0x7f34540d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 --2- 172.21.15.31:0/2925963722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b8ce0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 -- 172.21.15.31:0/2925963722 >> 172.21.15.31:0/2925963722 conn(0x7f34540720e0 msgr2=0x7f34540724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 -- 172.21.15.31:0/2925963722 shutdown_connections 2024-01-19T04:55:05.090 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.089+0000 7f3458a89640 1 -- 172.21.15.31:0/2925963722 wait complete. 2024-01-19T04:55:05.092 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.090+0000 7f3458a89640 1 Processor -- start 2024-01-19T04:55:05.092 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.090+0000 7f3458a89640 1 -- start start 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3458a89640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34540d1400 0x7f345414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3458a89640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d32e0 0x7f34540c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3458a89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3458a89640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34541f2e80 con 0x7f345415d540 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3458a89640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34540c8700 con 0x7f34540d1400 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3458a89640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34540c89e0 con 0x7f34540d32e0 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3452d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3452d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59464/0 (socket says 172.21.15.31:59464) 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.091+0000 7f3452d9d640 1 -- 172.21.15.31:0/3395451008 learned_addr learned my addr 172.21.15.31:0/3395451008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f344bfff640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d32e0 0x7f34540c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f345259c640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34540d1400 0x7f345414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3452d9d640 1 -- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d32e0 msgr2=0x7f34540c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3452d9d640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d32e0 0x7f34540c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.093 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3452d9d640 1 -- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34540d1400 msgr2=0x7f345414aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3452d9d640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34540d1400 0x7f345414aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3452d9d640 1 -- 172.21.15.31:0/3395451008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444009400 con 0x7f345415d540 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f344bfff640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d32e0 0x7f34540c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f345259c640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34540d1400 0x7f345414aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3452d9d640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b7a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f344c00c800 tx=0x7f344c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f344affd640 1 -- 172.21.15.31:0/3395451008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c004d60 con 0x7f345415d540 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f344affd640 1 -- 172.21.15.31:0/3395451008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c00a640 con 0x7f345415d540 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f344affd640 1 -- 172.21.15.31:0/3395451008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c00aa30 con 0x7f345415d540 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34540c8cc0 con 0x7f345415d540 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.092+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34540c9280 con 0x7f345415d540 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.093+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34540c94a0 con 0x7f345415d540 2024-01-19T04:55:05.094 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.093+0000 7f344affd640 1 -- 172.21.15.31:0/3395451008 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 89142+0+0 (secure 0 0 0) 0x7f344c00ab90 con 0x7f345415d540 2024-01-19T04:55:05.095 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.094+0000 7f344affd640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406ae40 0x7f343406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:05.095 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.094+0000 7f344affd640 1 -- 172.21.15.31:0/3395451008 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f344c087c10 con 0x7f345415d540 2024-01-19T04:55:05.095 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.094+0000 7f345259c640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406ae40 0x7f343406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:05.095 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.095+0000 7f345259c640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406ae40 0x7f343406d300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3444036000 tx=0x7f344403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:05.098 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.098+0000 7f344affd640 1 -- 172.21.15.31:0/3395451008 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f344c053a20 con 0x7f345415d540 2024-01-19T04:55:05.277 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.278+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34540c9ce0 con 0x7f345415d540 2024-01-19T04:55:05.278 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.278+0000 7f344affd640 1 -- 172.21.15.31:0/3395451008 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f344c018070 con 0x7f345415d540 2024-01-19T04:55:05.278 INFO:teuthology.misc.health.smithi031.stdout: 2024-01-19T04:55:05.278 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":12,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:02.558620+0000","last_up_change":"0.000000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:05.280 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406ae40 msgr2=0x7f343406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:05.280 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406ae40 0x7f343406d300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3444036000 tx=0x7f344403c040 comp rx=0 tx=0).stop 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 msgr2=0x7f34540b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b7a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f344c00c800 tx=0x7f344c00ccc0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 shutdown_connections 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406ae40 0x7f343406d300 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345415d540 0x7f34540b7a10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34540d32e0 0x7f34540c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.280+0000 7f3458a89640 1 --2- 172.21.15.31:0/3395451008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34540d1400 0x7f345414aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.281+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 >> 172.21.15.31:0/3395451008 conn(0x7f34540720e0 msgr2=0x7f345415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.281+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 shutdown_connections 2024-01-19T04:55:05.281 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:05.281+0000 7f3458a89640 1 -- 172.21.15.31:0/3395451008 wait complete. 2024-01-19T04:55:05.289 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-19T04:55:12.096 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:12.197 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.197+0000 7f0c3961d640 1 Processor -- start 2024-01-19T04:55:12.197 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.197+0000 7f0c3961d640 1 -- start start 2024-01-19T04:55:12.197 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.198+0000 7f0c3961d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f0c34141b20 0x7f0c3406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.198 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.198+0000 7f0c3961d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c340ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.198 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.198+0000 7f0c3961d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c34141f00 con 0x7f0c3406ccf0 2024-01-19T04:55:12.198 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.198+0000 7f0c3961d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c340b3390 con 0x7f0c34141b20 2024-01-19T04:55:12.198 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.198+0000 7f0c3961d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340b3630 con 0x7f0c340b13d0 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c337fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c340ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c32ffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f0c34141b20 0x7f0c3406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c32ffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f0c34141b20 0x7f0c3406c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:57426/0 (socket says 172.21.15.31:57426) 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c327fc640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f0c3406ccf0 0x7f0c340b0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:44852/0 (socket says 172.21.15.31:44852) 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c337fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c340ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:55628/0 (socket says 172.21.15.31:55628) 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c32ffd640 1 -- 172.21.15.31:0/3768051084 learned_addr learned my addr 172.21.15.31:0/3768051084 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c32ffd640 1 -- 172.21.15.31:0/3768051084 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 msgr2=0x7f0c340ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c32ffd640 1 --2- 172.21.15.31:0/3768051084 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c340ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c32ffd640 1 -- 172.21.15.31:0/3768051084 >> v1:172.21.15.137:6789/0 conn(0x7f0c3406ccf0 legacy=0x7f0c340b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.199 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.199+0000 7f0c32ffd640 1 -- 172.21.15.31:0/3768051084 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c340b3910 con 0x7f0c34141b20 2024-01-19T04:55:12.200 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.200+0000 7f0c32ffd640 1 --2- 172.21.15.31:0/3768051084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 0x7f0c3406c7b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009870 tx=0x7f0c2c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7680cd7e2a99b78 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.200 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.200+0000 7f0c31ffb640 1 -- 172.21.15.31:0/3768051084 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c031640 con 0x7f0c34141b20 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.200+0000 7f0c31ffb640 1 -- 172.21.15.31:0/3768051084 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0317a0 con 0x7f0c34141b20 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.200+0000 7f0c31ffb640 1 -- 172.21.15.31:0/3768051084 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c031b30 con 0x7f0c34141b20 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.200+0000 7f0c3961d640 1 -- 172.21.15.31:0/3768051084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 msgr2=0x7f0c3406c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.200+0000 7f0c3961d640 1 --2- 172.21.15.31:0/3768051084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 0x7f0c3406c7b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009870 tx=0x7f0c2c02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.200+0000 7f0c3961d640 1 -- 172.21.15.31:0/3768051084 shutdown_connections 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.201+0000 7f0c3961d640 1 --2- 172.21.15.31:0/3768051084 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c340ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.201+0000 7f0c3961d640 1 --2- 172.21.15.31:0/3768051084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 0x7f0c3406c7b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.201+0000 7f0c3961d640 1 -- 172.21.15.31:0/3768051084 >> 172.21.15.31:0/3768051084 conn(0x7f0c3406dee0 msgr2=0x7f0c3406a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.201+0000 7f0c3961d640 1 -- 172.21.15.31:0/3768051084 shutdown_connections 2024-01-19T04:55:12.201 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.201+0000 7f0c3961d640 1 -- 172.21.15.31:0/3768051084 wait complete. 2024-01-19T04:55:12.202 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.202+0000 7f0c3961d640 1 Processor -- start 2024-01-19T04:55:12.202 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.202+0000 7f0c3961d640 1 -- start start 2024-01-19T04:55:12.203 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c3961d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3406ccf0 0x7f0c34140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c3961d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c341f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c3961d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 0x7f0c34071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c3961d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34070640 con 0x7f0c3406ccf0 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c3961d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c341f38c0 con 0x7f0c34141b20 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c3961d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c341f3ba0 con 0x7f0c340b13d0 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3406ccf0 0x7f0c34140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3406ccf0 0x7f0c34140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33118/0 (socket says 172.21.15.31:33118) 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 -- 172.21.15.31:0/83933069 learned_addr learned my addr 172.21.15.31:0/83933069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c337fe640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 0x7f0c34071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c327fc640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c341f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 -- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 msgr2=0x7f0c341f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c341f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 -- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 msgr2=0x7f0c34071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.204 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 0x7f0c34071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.205 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 -- 172.21.15.31:0/83933069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0094d0 con 0x7f0c3406ccf0 2024-01-19T04:55:12.205 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.203+0000 7f0c32ffd640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3406ccf0 0x7f0c34140b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.205 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.204+0000 7f0c2b7fe640 1 -- 172.21.15.31:0/83933069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c03a930 con 0x7f0c3406ccf0 2024-01-19T04:55:12.205 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.204+0000 7f0c2b7fe640 1 -- 172.21.15.31:0/83933069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c03aa90 con 0x7f0c3406ccf0 2024-01-19T04:55:12.205 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.204+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c341f3e80 con 0x7f0c3406ccf0 2024-01-19T04:55:12.205 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.204+0000 7f0c2b7fe640 1 -- 172.21.15.31:0/83933069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c03ae40 con 0x7f0c3406ccf0 2024-01-19T04:55:12.205 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.204+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c341f4160 con 0x7f0c3406ccf0 2024-01-19T04:55:12.206 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.205+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c34070780 con 0x7f0c3406ccf0 2024-01-19T04:55:12.206 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.205+0000 7f0c2b7fe640 1 -- 172.21.15.31:0/83933069 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0c2c02f050 con 0x7f0c3406ccf0 2024-01-19T04:55:12.206 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.205+0000 7f0c2b7fe640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406aed0 0x7f0c1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.206 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.206+0000 7f0c2b7fe640 1 -- 172.21.15.31:0/83933069 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0c2c0ab080 con 0x7f0c3406ccf0 2024-01-19T04:55:12.206 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.206+0000 7f0c327fc640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406aed0 0x7f0c1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.206 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.206+0000 7f0c327fc640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406aed0 0x7f0c1406d390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c34155b00 tx=0x7f0c1c007ce0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.209 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.210+0000 7f0c2b7fe640 1 -- 172.21.15.31:0/83933069 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0c2c076330 con 0x7f0c3406ccf0 2024-01-19T04:55:12.390 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.391+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c3406d6a0 con 0x7f0c3406ccf0 2024-01-19T04:55:12.391 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.392+0000 7f0c2b7fe640 1 -- 172.21.15.31:0/83933069 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12815 (secure 0 0 0) 0x7f0c2c079950 con 0x7f0c3406ccf0 2024-01-19T04:55:12.391 INFO:teuthology.misc.health.smithi031.stdout: 2024-01-19T04:55:12.391 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":16,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:08.589531+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:12.393 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406aed0 msgr2=0x7f0c1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406aed0 0x7f0c1406d390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c34155b00 tx=0x7f0c1c007ce0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3406ccf0 msgr2=0x7f0c34140b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3406ccf0 0x7f0c34140b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c0046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 shutdown_connections 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406aed0 0x7f0c1406d390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c34141b20 0x7f0c34071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c340b13d0 0x7f0c341f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 --2- 172.21.15.31:0/83933069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3406ccf0 0x7f0c34140b90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 >> 172.21.15.31:0/83933069 conn(0x7f0c3406dee0 msgr2=0x7f0c3415e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 shutdown_connections 2024-01-19T04:55:12.394 INFO:teuthology.misc.health.smithi031.stderr:2024-01-19T04:55:12.394+0000 7f0c3961d640 1 -- 172.21.15.31:0/83933069 wait complete. 2024-01-19T04:55:12.403 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-19T04:55:12.403 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-19T04:55:12.403 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-19T04:55:12.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-19T04:55:12.405 INFO:tasks.ceph.mds.a:Started 2024-01-19T04:55:12.405 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-19T04:55:12.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-19T04:55:12.407 INFO:tasks.ceph.mds.c:Started 2024-01-19T04:55:12.407 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-19T04:55:12.408 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-19T04:55:12.410 INFO:tasks.ceph.mds.b:Started 2024-01-19T04:55:12.410 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-19T04:55:12.411 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-19T04:55:12.412 INFO:tasks.ceph.mds.d:Started 2024-01-19T04:55:12.413 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-19T04:55:12.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:12.481 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-01-19T04:55:12.482 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2024-01-19T04:55:12.496 INFO:tasks.ceph.mds.b.smithi137.stdout:starting mds.b at 2024-01-19T04:55:12.496 INFO:tasks.ceph.mds.d.smithi137.stdout:starting mds.d at 2024-01-19T04:55:12.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.520+0000 7fd8160a6640 1 Processor -- start 2024-01-19T04:55:12.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.520+0000 7fd8160a6640 1 -- start start 2024-01-19T04:55:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd8160a6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8100b3030 0x7fd810160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd8160a6640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd810071df0 0x7fd8101614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd8160a6640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8100b3510 con 0x7fd81015d300 2024-01-19T04:55:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd8160a6640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8100b37f0 con 0x7fd810071df0 2024-01-19T04:55:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd8160a6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100b3ad0 con 0x7fd8100b3030 2024-01-19T04:55:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd80effd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8100b3030 0x7fd810160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd80ffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd810071df0 0x7fd8101614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd80effd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8100b3030 0x7fd810160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:55660/0 (socket says 172.21.15.31:55660) 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.521+0000 7fd80effd640 1 -- 172.21.15.31:0/3136809915 learned_addr learned my addr 172.21.15.31:0/3136809915 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.522+0000 7fd80ffff640 1 -- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8100b3030 msgr2=0x7fd810160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.522+0000 7fd80ffff640 1 --2- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8100b3030 0x7fd810160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.522+0000 7fd80ffff640 1 -- 172.21.15.31:0/3136809915 >> v1:172.21.15.137:6789/0 conn(0x7fd81015d300 legacy=0x7fd81006ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.522+0000 7fd80ffff640 1 -- 172.21.15.31:0/3136809915 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81006a650 con 0x7fd810071df0 2024-01-19T04:55:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.522+0000 7fd80effd640 1 --2- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8100b3030 0x7fd810160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:12.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.522+0000 7fd80ffff640 1 --2- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd810071df0 0x7fd8101614a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd808009d90 tx=0x7fd80802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d7e34001b19b6bd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd80e7fc640 1 -- 172.21.15.31:0/3136809915 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80802cc50 con 0x7fd810071df0 2024-01-19T04:55:12.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd80e7fc640 1 -- 172.21.15.31:0/3136809915 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80802cdb0 con 0x7fd810071df0 2024-01-19T04:55:12.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd80e7fc640 1 -- 172.21.15.31:0/3136809915 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808034660 con 0x7fd810071df0 2024-01-19T04:55:12.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 -- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd810071df0 msgr2=0x7fd8101614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd810071df0 0x7fd8101614a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd808009d90 tx=0x7fd80802bea0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 -- 172.21.15.31:0/3136809915 shutdown_connections 2024-01-19T04:55:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd810071df0 0x7fd8101614a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3136809915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8100b3030 0x7fd810160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 -- 172.21.15.31:0/3136809915 >> 172.21.15.31:0/3136809915 conn(0x7fd81006d750 msgr2=0x7fd81006db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 -- 172.21.15.31:0/3136809915 shutdown_connections 2024-01-19T04:55:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.523+0000 7fd8160a6640 1 -- 172.21.15.31:0/3136809915 wait complete. 2024-01-19T04:55:12.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.524+0000 7fd8160a6640 1 Processor -- start 2024-01-19T04:55:12.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.524+0000 7fd8160a6640 1 -- start start 2024-01-19T04:55:12.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd8160a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd810071df0 0x7fd8101532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd8160a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8100b3030 0x7fd81015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd8160a6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd81015d300 0x7fd810146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd8160a6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81015df20 con 0x7fd810071df0 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd8160a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd810146bd0 con 0x7fd8100b3030 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd8160a6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81015e110 con 0x7fd81015d300 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd810071df0 0x7fd8101532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd810071df0 0x7fd8101532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33158/0 (socket says 172.21.15.31:33158) 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd80f7fe640 1 -- 172.21.15.31:0/3099478650 learned_addr learned my addr 172.21.15.31:0/3099478650 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd80ffff640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd81015d300 0x7fd810146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.525+0000 7fd80effd640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8100b3030 0x7fd81015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd80f7fe640 1 -- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd81015d300 msgr2=0x7fd810146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd80f7fe640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd81015d300 0x7fd810146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd80f7fe640 1 -- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8100b3030 msgr2=0x7fd81015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd80f7fe640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8100b3030 0x7fd81015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd80f7fe640 1 -- 172.21.15.31:0/3099478650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040094d0 con 0x7fd810071df0 2024-01-19T04:55:12.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd80f7fe640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd810071df0 0x7fd8101532a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c3c0 tx=0x7fd80000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd7f7fff640 1 -- 172.21.15.31:0/3099478650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000052d0 con 0x7fd810071df0 2024-01-19T04:55:12.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd7f7fff640 1 -- 172.21.15.31:0/3099478650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd800005450 con 0x7fd810071df0 2024-01-19T04:55:12.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd808009a70 con 0x7fd810071df0 2024-01-19T04:55:12.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd7f7fff640 1 -- 172.21.15.31:0/3099478650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800007260 con 0x7fd810071df0 2024-01-19T04:55:12.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.526+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81015e6d0 con 0x7fd810071df0 2024-01-19T04:55:12.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.527+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8100696e0 con 0x7fd810071df0 2024-01-19T04:55:12.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.528+0000 7fd7f7fff640 1 -- 172.21.15.31:0/3099478650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd8000073c0 con 0x7fd810071df0 2024-01-19T04:55:12.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.528+0000 7fd7f7fff640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7fc06aed0 0x7fd7fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.528+0000 7fd7f7fff640 1 -- 172.21.15.31:0/3099478650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd800088b60 con 0x7fd810071df0 2024-01-19T04:55:12.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.528+0000 7fd80effd640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7fc06aed0 0x7fd7fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.529+0000 7fd80effd640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7fc06aed0 0x7fd7fc06d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd810158910 tx=0x7fd804009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.532+0000 7fd7f7fff640 1 -- 172.21.15.31:0/3099478650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd800053e10 con 0x7fd810071df0 2024-01-19T04:55:12.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.711+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd810071bd0 con 0x7fd810071df0 2024-01-19T04:55:12.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.711+0000 7fd7f7fff640 1 -- 172.21.15.31:0/3099478650 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12815 (secure 0 0 0) 0x7fd800020090 con 0x7fd810071df0 2024-01-19T04:55:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:12.711 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:08.589531+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7fc06aed0 msgr2=0x7fd7fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7fc06aed0 0x7fd7fc06d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd810158910 tx=0x7fd804009240 comp rx=0 tx=0).stop 2024-01-19T04:55:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd810071df0 msgr2=0x7fd8101532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd810071df0 0x7fd8101532a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c3c0 tx=0x7fd80000c880 comp rx=0 tx=0).stop 2024-01-19T04:55:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 shutdown_connections 2024-01-19T04:55:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7fc06aed0 0x7fd7fc06d390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd81015d300 0x7fd810146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8100b3030 0x7fd81015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 --2- 172.21.15.31:0/3099478650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd810071df0 0x7fd8101532a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 >> 172.21.15.31:0/3099478650 conn(0x7fd81006d750 msgr2=0x7fd81006db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:12.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 shutdown_connections 2024-01-19T04:55:12.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.713+0000 7fd8160a6640 1 -- 172.21.15.31:0/3099478650 wait complete. 2024-01-19T04:55:12.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:12.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:12.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.827+0000 7fd015889640 1 Processor -- start 2024-01-19T04:55:12.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.827+0000 7fd015889640 1 -- start start 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd015889640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd01006f900 0x7fd010071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd015889640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd01006a5b0 0x7fd0100cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd015889640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01015d9d0 con 0x7fd0100cc1e0 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd015889640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01006fce0 con 0x7fd01006f900 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd015889640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01006aa40 con 0x7fd01006a5b0 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd00effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd01006f900 0x7fd010071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd00f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd0100cc1e0 0x7fd0101610c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33026/0 (socket says 172.21.15.31:33026) 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.828+0000 7fd00f7fe640 1 -- 172.21.15.31:0/281833765 learned_addr learned my addr 172.21.15.31:0/281833765 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00e7fc640 1 --2- 172.21.15.31:0/281833765 >> v2:172.21.15.137:3300/0 conn(0x7fd01006a5b0 0x7fd0100cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00effd640 1 -- 172.21.15.31:0/281833765 >> v2:172.21.15.137:3300/0 conn(0x7fd01006a5b0 msgr2=0x7fd0100cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00effd640 1 --2- 172.21.15.31:0/281833765 >> v2:172.21.15.137:3300/0 conn(0x7fd01006a5b0 0x7fd0100cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00effd640 1 -- 172.21.15.31:0/281833765 >> v1:172.21.15.31:6789/0 conn(0x7fd0100cc1e0 legacy=0x7fd0101610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00dffb640 1 -- 172.21.15.31:0/281833765 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045066125 0 0) 0x7fd01015d9d0 con 0x7fd0100cc1e0 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00effd640 1 -- 172.21.15.31:0/281833765 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01006ad20 con 0x7fd01006f900 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00e7fc640 1 --2- 172.21.15.31:0/281833765 >> v2:172.21.15.137:3300/0 conn(0x7fd01006a5b0 0x7fd0100cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00effd640 1 --2- 172.21.15.31:0/281833765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01006f900 0x7fd010071e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd010066f90 tx=0x7fd00802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ff8b1e820f89ed server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00dffb640 1 -- 172.21.15.31:0/281833765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd008031830 con 0x7fd01006f900 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.829+0000 7fd00dffb640 1 -- 172.21.15.31:0/281833765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd008031990 con 0x7fd01006f900 2024-01-19T04:55:12.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd00dffb640 1 -- 172.21.15.31:0/281833765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd008031d20 con 0x7fd01006f900 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 -- 172.21.15.31:0/281833765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01006f900 msgr2=0x7fd010071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 --2- 172.21.15.31:0/281833765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01006f900 0x7fd010071e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd010066f90 tx=0x7fd00802b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 -- 172.21.15.31:0/281833765 shutdown_connections 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 --2- 172.21.15.31:0/281833765 >> v2:172.21.15.137:3300/0 conn(0x7fd01006a5b0 0x7fd0100cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 --2- 172.21.15.31:0/281833765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01006f900 0x7fd010071e40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 -- 172.21.15.31:0/281833765 >> 172.21.15.31:0/281833765 conn(0x7fd0100d0000 msgr2=0x7fd0100d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 -- 172.21.15.31:0/281833765 shutdown_connections 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.830+0000 7fd015889640 1 -- 172.21.15.31:0/281833765 wait complete. 2024-01-19T04:55:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.831+0000 7fd015889640 1 Processor -- start 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.831+0000 7fd015889640 1 -- start start 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd015889640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd01006f900 0x7fd0101f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd015889640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0100cc1e0 0x7fd0100b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd015889640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd010161490 0x7fd0101f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd015889640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0101f6880 con 0x7fd010161490 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd015889640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100a1400 con 0x7fd0100cc1e0 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd015889640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100a0880 con 0x7fd01006f900 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd010161490 0x7fd0101f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd010161490 0x7fd0101f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33166/0 (socket says 172.21.15.31:33166) 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 -- 172.21.15.31:0/64934898 learned_addr learned my addr 172.21.15.31:0/64934898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00effd640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd01006f900 0x7fd0101f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00e7fc640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0100cc1e0 0x7fd0100b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 -- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd01006f900 msgr2=0x7fd0101f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd01006f900 0x7fd0101f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 -- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0100cc1e0 msgr2=0x7fd0100b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0100cc1e0 0x7fd0100b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00f7fe640 1 -- 172.21.15.31:0/64934898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd008009400 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00effd640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd01006f900 0x7fd0101f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.832+0000 7fd00e7fc640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0100cc1e0 0x7fd0100b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.833+0000 7fd00f7fe640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd010161490 0x7fd0101f2180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c700 tx=0x7fd00000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.833+0000 7fd007fff640 1 -- 172.21.15.31:0/64934898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000ce10 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.833+0000 7fd007fff640 1 -- 172.21.15.31:0/64934898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00000a640 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.833+0000 7fd007fff640 1 -- 172.21.15.31:0/64934898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000aa70 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.833+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0100a0b60 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.833+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0100a1120 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.834+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0101509a0 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.834+0000 7fd007fff640 1 -- 172.21.15.31:0/64934898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd00000abd0 con 0x7fd010161490 2024-01-19T04:55:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.834+0000 7fd007fff640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcff006af00 0x7fcff006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.835+0000 7fd007fff640 1 -- 172.21.15.31:0/64934898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0000889d0 con 0x7fd010161490 2024-01-19T04:55:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.835+0000 7fd00effd640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcff006af00 0x7fcff006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.835+0000 7fd00effd640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcff006af00 0x7fcff006d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd008036000 tx=0x7fd008008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:12.839+0000 7fd007fff640 1 -- 172.21.15.31:0/64934898 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd000053bf0 con 0x7fd010161490 2024-01-19T04:55:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.015+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd01014c6e0 con 0x7fd010161490 2024-01-19T04:55:13.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.015+0000 7fd007fff640 1 -- 172.21.15.31:0/64934898 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd01014c6e0 con 0x7fd010161490 2024-01-19T04:55:13.015 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcff006af00 msgr2=0x7fcff006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcff006af00 0x7fcff006d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd008036000 tx=0x7fd008008030 comp rx=0 tx=0).stop 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd010161490 msgr2=0x7fd0101f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd010161490 0x7fd0101f2180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c700 tx=0x7fd00000cbc0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 shutdown_connections 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcff006af00 0x7fcff006d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd010161490 0x7fd0101f2180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0100cc1e0 0x7fd0100b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 --2- 172.21.15.31:0/64934898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd01006f900 0x7fd0101f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 >> 172.21.15.31:0/64934898 conn(0x7fd0100d0000 msgr2=0x7fd0100cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:13.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.017+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 shutdown_connections 2024-01-19T04:55:13.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.018+0000 7fd015889640 1 -- 172.21.15.31:0/64934898 wait complete. 2024-01-19T04:55:13.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:55:13.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.129+0000 7f90ec566640 1 Processor -- start 2024-01-19T04:55:13.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.129+0000 7f90ec566640 1 -- start start 2024-01-19T04:55:13.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.130+0000 7f90ec566640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e40d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:13.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.130+0000 7f90ec566640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f90e40cee00 0x7f90e415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:13.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.130+0000 7f90ec566640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e40d0040 con 0x7f90e406ecb0 2024-01-19T04:55:13.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.130+0000 7f90ec566640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e415c800 con 0x7f90e40cee00 2024-01-19T04:55:13.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.130+0000 7f90ec566640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40cd500 con 0x7f90e406e8d0 2024-01-19T04:55:13.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90ea302640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e40d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:13.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90e9b01640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f90e40cee00 0x7f90e415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:13.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90e9b01640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f90e40cee00 0x7f90e415c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:57498/0 (socket says 172.21.15.31:57498) 2024-01-19T04:55:13.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90eab03640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f90e406ecb0 0x7f90e40cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:44880/0 (socket says 172.21.15.31:44880) 2024-01-19T04:55:13.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90e9b01640 1 -- 172.21.15.31:0/1510248843 learned_addr learned my addr 172.21.15.31:0/1510248843 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:13.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90ea302640 1 --2- 172.21.15.31:0/1510248843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e40d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:55700/0 (socket says 172.21.15.31:55700) 2024-01-19T04:55:13.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90ea302640 1 -- 172.21.15.31:0/1510248843 >> v2:172.21.15.137:3300/0 conn(0x7f90e40cee00 msgr2=0x7f90e415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:13.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90ea302640 1 --2- 172.21.15.31:0/1510248843 >> v2:172.21.15.137:3300/0 conn(0x7f90e40cee00 0x7f90e415c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90ea302640 1 -- 172.21.15.31:0/1510248843 >> v1:172.21.15.137:6789/0 conn(0x7f90e406ecb0 legacy=0x7f90e40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90e9300640 1 -- 172.21.15.31:0/1510248843 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59666047 0 0) 0x7f90e40d0040 con 0x7f90e406ecb0 2024-01-19T04:55:13.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90ea302640 1 -- 172.21.15.31:0/1510248843 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e40cb800 con 0x7f90e406e8d0 2024-01-19T04:55:13.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.131+0000 7f90e9b01640 1 --2- 172.21.15.31:0/1510248843 >> v2:172.21.15.137:3300/0 conn(0x7f90e40cee00 0x7f90e415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:13.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.132+0000 7f90ea302640 1 --2- 172.21.15.31:0/1510248843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e40d08a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009d90 tx=0x7f90d402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f763a82f821169 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:13.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.132+0000 7f90e9300640 1 -- 172.21.15.31:0/1510248843 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d402cbc0 con 0x7f90e406e8d0 2024-01-19T04:55:13.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.132+0000 7f90e9300640 1 -- 172.21.15.31:0/1510248843 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d402cd20 con 0x7f90e406e8d0 2024-01-19T04:55:13.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.132+0000 7f90e9300640 1 -- 172.21.15.31:0/1510248843 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4034660 con 0x7f90e406e8d0 2024-01-19T04:55:13.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 -- 172.21.15.31:0/1510248843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 msgr2=0x7f90e40d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 --2- 172.21.15.31:0/1510248843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e40d08a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009d90 tx=0x7f90d402be10 comp rx=0 tx=0).stop 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 -- 172.21.15.31:0/1510248843 shutdown_connections 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 --2- 172.21.15.31:0/1510248843 >> v2:172.21.15.137:3300/0 conn(0x7f90e40cee00 0x7f90e415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 --2- 172.21.15.31:0/1510248843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e40d08a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 -- 172.21.15.31:0/1510248843 >> 172.21.15.31:0/1510248843 conn(0x7f90e40b2c60 msgr2=0x7f90e40b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 -- 172.21.15.31:0/1510248843 shutdown_connections 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.133+0000 7f90ec566640 1 -- 172.21.15.31:0/1510248843 wait complete. 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.134+0000 7f90ec566640 1 Processor -- start 2024-01-19T04:55:13.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.134+0000 7f90ec566640 1 -- start start 2024-01-19T04:55:13.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90ec566640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e41573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:13.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90ec566640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e406ecb0 0x7f90e40c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:13.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90ec566640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e40cee00 0x7f90e40c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:13.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90ec566640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e4157920 con 0x7f90e40cee00 2024-01-19T04:55:13.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90ec566640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40b57b0 con 0x7f90e406ecb0 2024-01-19T04:55:13.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90ec566640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c5f00 con 0x7f90e406e8d0 2024-01-19T04:55:13.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e40cee00 0x7f90e40c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e40cee00 0x7f90e40c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33168/0 (socket says 172.21.15.31:33168) 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 -- 172.21.15.31:0/2214520883 learned_addr learned my addr 172.21.15.31:0/2214520883 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90ea302640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e41573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90e9b01640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e406ecb0 0x7f90e40c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 -- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 msgr2=0x7f90e41573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e41573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 -- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e406ecb0 msgr2=0x7f90e40c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e406ecb0 0x7f90e40c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 -- 172.21.15.31:0/2214520883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d80094d0 con 0x7f90e40cee00 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90eab03640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e40cee00 0x7f90e40c5180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f90dc001eb0 tx=0x7f90dc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90e2ffd640 1 -- 172.21.15.31:0/2214520883 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0074e0 con 0x7f90e40cee00 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.135+0000 7f90e2ffd640 1 -- 172.21.15.31:0/2214520883 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc007640 con 0x7f90e40cee00 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.136+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d4009a70 con 0x7f90e40cee00 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.136+0000 7f90e2ffd640 1 -- 172.21.15.31:0/2214520883 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc004a00 con 0x7f90e40cee00 2024-01-19T04:55:13.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.136+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e40b5ce0 con 0x7f90e40cee00 2024-01-19T04:55:13.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.136+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e4074a80 con 0x7f90e40cee00 2024-01-19T04:55:13.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.137+0000 7f90e2ffd640 1 -- 172.21.15.31:0/2214520883 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f90dc004b60 con 0x7f90e40cee00 2024-01-19T04:55:13.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.137+0000 7f90e2ffd640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f90d006aed0 0x7f90d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:13.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.137+0000 7f90e2ffd640 1 -- 172.21.15.31:0/2214520883 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f90dc088f50 con 0x7f90e40cee00 2024-01-19T04:55:13.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.137+0000 7f90ea302640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f90d006aed0 0x7f90d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:13.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.138+0000 7f90ea302640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f90d006aed0 0x7f90d006d390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f90d4036000 tx=0x7f90d4030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.142+0000 7f90e2ffd640 1 -- 172.21.15.31:0/2214520883 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f90dc057d40 con 0x7f90e40cee00 2024-01-19T04:55:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.361+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90e40b67c0 con 0x7f90e40cee00 2024-01-19T04:55:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.361+0000 7f90e2ffd640 1 -- 172.21.15.31:0/2214520883 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f90dc0578d0 con 0x7f90e40cee00 2024-01-19T04:55:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:13.361 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-19T04:55:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f90d006aed0 msgr2=0x7f90d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f90d006aed0 0x7f90d006d390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f90d4036000 tx=0x7f90d4030000 comp rx=0 tx=0).stop 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e40cee00 msgr2=0x7f90e40c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e40cee00 0x7f90e40c5180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f90dc001eb0 tx=0x7f90dc00c870 comp rx=0 tx=0).stop 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 shutdown_connections 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f90d006aed0 0x7f90d006d390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e40cee00 0x7f90e40c5180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e406ecb0 0x7f90e40c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 --2- 172.21.15.31:0/2214520883 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90e406e8d0 0x7f90e41573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 >> 172.21.15.31:0/2214520883 conn(0x7f90e40b2c60 msgr2=0x7f90e40d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 shutdown_connections 2024-01-19T04:55:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:13.363+0000 7f90ec566640 1 -- 172.21.15.31:0/2214520883 wait complete. 2024-01-19T04:55:13.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:55:13.488 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:55:15.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:55:15.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.487+0000 7f5d11a1e640 1 Processor -- start 2024-01-19T04:55:15.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.487+0000 7f5d11a1e640 1 -- start start 2024-01-19T04:55:15.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d11a1e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 0x7f5d0c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d11a1e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5d0c0d1600 0x7f5d0c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d11a1e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c0d06b0 con 0x7f5d0c0cfd10 2024-01-19T04:55:15.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d11a1e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c0b66b0 con 0x7f5d0c0d1600 2024-01-19T04:55:15.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d11a1e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0d1b90 con 0x7f5d0c160bb0 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d0a7fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5d0c0cfd10 0x7f5d0c0d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33030/0 (socket says 172.21.15.31:33030) 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d0a7fc640 1 -- 172.21.15.31:0/2188936571 learned_addr learned my addr 172.21.15.31:0/2188936571 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751096650 0 0) 0x7f5d0c0d06b0 con 0x7f5d0c0cfd10 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf4003520 con 0x7f5d0c0cfd10 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d0affd640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 0x7f5d0c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.488+0000 7f5d0b7fe640 1 --2- 172.21.15.31:0/2188936571 >> v2:172.21.15.137:3300/0 conn(0x7f5d0c0d1600 0x7f5d0c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101536868 0 0) 0x7f5cf4003520 con 0x7f5d0c0cfd10 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 msgr2=0x7f5d0c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 0x7f5d0c0b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 >> v2:172.21.15.137:3300/0 conn(0x7f5d0c0d1600 msgr2=0x7f5d0c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 --2- 172.21.15.31:0/2188936571 >> v2:172.21.15.137:3300/0 conn(0x7f5d0c0d1600 0x7f5d0c0d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0d1e70 con 0x7f5d0c0cfd10 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f5d00001670 con 0x7f5d0c0cfd10 2024-01-19T04:55:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 >> v1:172.21.15.31:6789/0 conn(0x7f5d0c0cfd10 legacy=0x7f5d0c0d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 0x7f5cf4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5cf40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf4004d00 0x7f5cf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0b66b0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0d1b90 con 0x7f5cf4004d00 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf400b980 con 0x7f5d0c160bb0 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5cf40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0affd640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 0x7f5cf4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0b7fe640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf4004d00 0x7f5cf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0a7fc640 1 -- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 msgr2=0x7f5cf4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c160bb0 0x7f5cf4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0a7fc640 1 -- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf4004d00 msgr2=0x7f5cf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf4004d00 0x7f5cf4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.489+0000 7f5d0a7fc640 1 -- 172.21.15.31:0/2188936571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4003520 con 0x7f5cf4003ef0 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.490+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5cf40047c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c210 tx=0x7f5cf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f928c885646a18cd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.490+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0002cae0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.490+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0002cc40 con 0x7f5cf4003ef0 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.490+0000 7f5d09ffb640 1 -- 172.21.15.31:0/2188936571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000058d0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.491+0000 7f5d11a1e640 1 -- 172.21.15.31:0/2188936571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 msgr2=0x7f5cf40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.491+0000 7f5d11a1e640 1 --2- 172.21.15.31:0/2188936571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5cf40047c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c210 tx=0x7f5cf40057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.491+0000 7f5d0b7fe640 1 -- 172.21.15.31:0/2188936571 reap_dead start 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.491+0000 7f5d11a1e640 1 -- 172.21.15.31:0/2188936571 shutdown_connections 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.491+0000 7f5d11a1e640 1 -- 172.21.15.31:0/2188936571 >> 172.21.15.31:0/2188936571 conn(0x7f5d0c09c600 msgr2=0x7f5d0c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.491+0000 7f5d11a1e640 1 -- 172.21.15.31:0/2188936571 shutdown_connections 2024-01-19T04:55:15.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.491+0000 7f5d11a1e640 1 -- 172.21.15.31:0/2188936571 wait complete. 2024-01-19T04:55:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.492+0000 7f5d11a1e640 1 Processor -- start 2024-01-19T04:55:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.492+0000 7f5d11a1e640 1 -- start start 2024-01-19T04:55:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d11a1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5d0c1f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d11a1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d0c153b90 0x7f5d0c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d11a1e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c153ff0 0x7f5d0c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d11a1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c1f0cd0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d11a1e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c7fd0 con 0x7f5d0c153b90 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d11a1e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c8220 con 0x7f5d0c153ff0 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5d0c1f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5d0c1f0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33192/0 (socket says 172.21.15.31:33192) 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 -- 172.21.15.31:0/3041849846 learned_addr learned my addr 172.21.15.31:0/3041849846 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d0c153b90 0x7f5d0c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0b7fe640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c153ff0 0x7f5d0c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 -- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c153ff0 msgr2=0x7f5d0c0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c153ff0 0x7f5d0c0c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 -- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d0c153b90 msgr2=0x7f5d0c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d0c153b90 0x7f5d0c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0affd640 1 -- 172.21.15.31:0/3041849846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0c8500 con 0x7f5cf4003ef0 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.493+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d0c153b90 0x7f5d0c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.494+0000 7f5d0affd640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5d0c1f0790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0119a0 tx=0x7f5cfc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.494+0000 7f5cf3fff640 1 -- 172.21.15.31:0/3041849846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00faf0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.494+0000 7f5cf3fff640 1 -- 172.21.15.31:0/3041849846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc00fc50 con 0x7f5cf4003ef0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.494+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c0c87e0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.494+0000 7f5cf3fff640 1 -- 172.21.15.31:0/3041849846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc014440 con 0x7f5cf4003ef0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.494+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c1f4010 con 0x7f5cf4003ef0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.495+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c0696e0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.495+0000 7f5cf3fff640 1 -- 172.21.15.31:0/3041849846 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5cfc01d400 con 0x7f5cf4003ef0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.496+0000 7f5cf3fff640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5cf806aed0 0x7f5cf806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.496+0000 7f5cf3fff640 1 -- 172.21.15.31:0/3041849846 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5cfc089a10 con 0x7f5cf4003ef0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.496+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5cf806aed0 0x7f5cf806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.496+0000 7f5d0a7fc640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5cf806aed0 0x7f5cf806d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d00034000 tx=0x7f5d00037040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:15.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.500+0000 7f5cf3fff640 1 -- 172.21.15.31:0/3041849846 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5cfc054c30 con 0x7f5cf4003ef0 2024-01-19T04:55:15.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.715+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d0c08d3b0 con 0x7f5cf4003ef0 2024-01-19T04:55:15.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.716+0000 7f5cf3fff640 1 -- 172.21.15.31:0/3041849846 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f5cfc058250 con 0x7f5cf4003ef0 2024-01-19T04:55:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:15.716 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-19T04:55:15.716 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5cf806aed0 msgr2=0x7f5cf806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5cf806aed0 0x7f5cf806d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d00034000 tx=0x7f5d00037040 comp rx=0 tx=0).stop 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 msgr2=0x7f5d0c1f0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5d0c1f0790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0119a0 tx=0x7f5cfc011e60 comp rx=0 tx=0).stop 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 shutdown_connections 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5cf806aed0 0x7f5cf806d390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d0c153ff0 0x7f5d0c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d0c153b90 0x7f5d0c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 --2- 172.21.15.31:0/3041849846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cf4003ef0 0x7f5d0c1f0790 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 >> 172.21.15.31:0/3041849846 conn(0x7f5d0c09c600 msgr2=0x7f5d0c073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 shutdown_connections 2024-01-19T04:55:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.718+0000 7f5d11a1e640 1 -- 172.21.15.31:0/3041849846 wait complete. 2024-01-19T04:55:15.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:15.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.829+0000 7f410d8c9640 1 Processor -- start 2024-01-19T04:55:15.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.829+0000 7f410d8c9640 1 -- start start 2024-01-19T04:55:15.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f410d8c9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4108160bb0 0x7f41080b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f410d8c9640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f41080cfd10 0x7f41080d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f410d8c9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41080d06b0 con 0x7f41080d0ef0 2024-01-19T04:55:15.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f410d8c9640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080b66b0 con 0x7f41080cfd10 2024-01-19T04:55:15.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f410d8c9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41080d1b90 con 0x7f4108160bb0 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f41077fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f41080d0ef0 0x7f41080d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33034/0 (socket says 172.21.15.31:33034) 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f41077fe640 1 -- 172.21.15.31:0/2379766700 learned_addr learned my addr 172.21.15.31:0/2379766700 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4106ffd640 1 --2- 172.21.15.31:0/2379766700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4108160bb0 0x7f41080b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f41067fc640 1 --2- 172.21.15.31:0/2379766700 >> v2:172.21.15.137:3300/0 conn(0x7f41080cfd10 0x7f41080d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4105ffb640 1 -- 172.21.15.31:0/2379766700 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741140720 0 0) 0x7f41080d06b0 con 0x7f41080d0ef0 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4105ffb640 1 -- 172.21.15.31:0/2379766700 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40f0003520 con 0x7f41080d0ef0 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4105ffb640 1 -- 172.21.15.31:0/2379766700 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2284573506 0 0) 0x7f40f0003520 con 0x7f41080d0ef0 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4106ffd640 1 -- 172.21.15.31:0/2379766700 >> v2:172.21.15.137:3300/0 conn(0x7f41080cfd10 msgr2=0x7f41080d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4106ffd640 1 --2- 172.21.15.31:0/2379766700 >> v2:172.21.15.137:3300/0 conn(0x7f41080cfd10 0x7f41080d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4106ffd640 1 -- 172.21.15.31:0/2379766700 >> v1:172.21.15.31:6789/0 conn(0x7f41080d0ef0 legacy=0x7f41080d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.830+0000 7f4106ffd640 1 -- 172.21.15.31:0/2379766700 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080d1e70 con 0x7f4108160bb0 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.831+0000 7f4106ffd640 1 --2- 172.21.15.31:0/2379766700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4108160bb0 0x7f41080b6170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f41000097a0 tx=0x7f410002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9051c0598d74c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.831+0000 7f4105ffb640 1 -- 172.21.15.31:0/2379766700 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80019f0 con 0x7f4108160bb0 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.831+0000 7f4105ffb640 1 -- 172.21.15.31:0/2379766700 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410002e050 con 0x7f4108160bb0 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.831+0000 7f4105ffb640 1 -- 172.21.15.31:0/2379766700 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100030710 con 0x7f4108160bb0 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 -- 172.21.15.31:0/2379766700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4108160bb0 msgr2=0x7f41080b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 --2- 172.21.15.31:0/2379766700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4108160bb0 0x7f41080b6170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f41000097a0 tx=0x7f410002b790 comp rx=0 tx=0).stop 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 -- 172.21.15.31:0/2379766700 shutdown_connections 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 --2- 172.21.15.31:0/2379766700 >> v2:172.21.15.137:3300/0 conn(0x7f41080cfd10 0x7f41080d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 --2- 172.21.15.31:0/2379766700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4108160bb0 0x7f41080b6170 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 -- 172.21.15.31:0/2379766700 >> 172.21.15.31:0/2379766700 conn(0x7f410809c600 msgr2=0x7f41080d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 -- 172.21.15.31:0/2379766700 shutdown_connections 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.832+0000 7f410d8c9640 1 -- 172.21.15.31:0/2379766700 wait complete. 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.833+0000 7f410d8c9640 1 Processor -- start 2024-01-19T04:55:15.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.833+0000 7f410d8c9640 1 -- start start 2024-01-19T04:55:15.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f410d8c9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41080cfd10 0x7f410815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f410d8c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41080d0ef0 0x7f410815f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f410d8c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4108160bb0 0x7f41081f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f410d8c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410815fec0 con 0x7f41080d0ef0 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f410d8c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410815f4f0 con 0x7f4108160bb0 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f410d8c9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41081f3f00 con 0x7f41080cfd10 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41080d0ef0 0x7f410815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41080d0ef0 0x7f410815f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33200/0 (socket says 172.21.15.31:33200) 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 -- 172.21.15.31:0/4264329497 learned_addr learned my addr 172.21.15.31:0/4264329497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f4106ffd640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41080cfd10 0x7f410815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41077fe640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4108160bb0 0x7f41081f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 -- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41080cfd10 msgr2=0x7f410815a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41080cfd10 0x7f410815a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 -- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4108160bb0 msgr2=0x7f41081f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4108160bb0 0x7f41081f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.834+0000 7f41067fc640 1 -- 172.21.15.31:0/4264329497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009400 con 0x7f41080d0ef0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.835+0000 7f41067fc640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41080d0ef0 0x7f410815f980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00e8a0 tx=0x7f40fc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.835+0000 7f40effff640 1 -- 172.21.15.31:0/4264329497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0094f0 con 0x7f41080d0ef0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.835+0000 7f40effff640 1 -- 172.21.15.31:0/4264329497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc00c940 con 0x7f41080d0ef0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.835+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410815d790 con 0x7f41080d0ef0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.835+0000 7f40effff640 1 -- 172.21.15.31:0/4264329497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc00ccf0 con 0x7f41080d0ef0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.835+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410815eea0 con 0x7f41080d0ef0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.836+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410815ed00 con 0x7f41080d0ef0 2024-01-19T04:55:15.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.836+0000 7f40effff640 1 -- 172.21.15.31:0/4264329497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f40fc013470 con 0x7f41080d0ef0 2024-01-19T04:55:15.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.837+0000 7f40effff640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40f406af00 0x7f40f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:15.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.837+0000 7f40effff640 1 -- 172.21.15.31:0/4264329497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40fc089400 con 0x7f41080d0ef0 2024-01-19T04:55:15.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.837+0000 7f4106ffd640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40f406af00 0x7f40f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:15.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.837+0000 7f4106ffd640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40f406af00 0x7f40f406d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4100034000 tx=0x7f410002f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:15.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:15.841+0000 7f40effff640 1 -- 172.21.15.31:0/4264329497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f410815ed00 con 0x7f41080d0ef0 2024-01-19T04:55:16.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.018+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f410815ed00 con 0x7f41080d0ef0 2024-01-19T04:55:16.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.019+0000 7f40effff640 1 -- 172.21.15.31:0/4264329497 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12815 (secure 0 0 0) 0x7f410815ed00 con 0x7f41080d0ef0 2024-01-19T04:55:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:16.019 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:08.589531+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40f406af00 msgr2=0x7f40f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40f406af00 0x7f40f406d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4100034000 tx=0x7f410002f000 comp rx=0 tx=0).stop 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41080d0ef0 msgr2=0x7f410815f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41080d0ef0 0x7f410815f980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00e8a0 tx=0x7f40fc00ed60 comp rx=0 tx=0).stop 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 shutdown_connections 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40f406af00 0x7f40f406d3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4108160bb0 0x7f41081f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41080d0ef0 0x7f410815f980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 --2- 172.21.15.31:0/4264329497 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41080cfd10 0x7f410815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 >> 172.21.15.31:0/4264329497 conn(0x7f410809c600 msgr2=0x7f410806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 shutdown_connections 2024-01-19T04:55:16.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.021+0000 7f410d8c9640 1 -- 172.21.15.31:0/4264329497 wait complete. 2024-01-19T04:55:16.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:16.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:16.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.135+0000 7fa3c86ee640 1 Processor -- start 2024-01-19T04:55:16.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.135+0000 7fa3c86ee640 1 -- start start 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.136+0000 7fa3c86ee640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa3c00cf6f0 0x7fa3c00add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.136+0000 7fa3c86ee640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa3c00cfad0 0x7fa3c00cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.136+0000 7fa3c86ee640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00ccc70 0x7fa3c00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.136+0000 7fa3c86ee640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c00cd0a0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.136+0000 7fa3c86ee640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c00b0a70 con 0x7fa3c00cfad0 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.136+0000 7fa3c86ee640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00b18d0 con 0x7fa3c00ccc70 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa3c00cf6f0 0x7fa3c00add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa3c00cf6f0 0x7fa3c00add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33204/0 (socket says 172.21.15.31:33204) 2024-01-19T04:55:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 -- 172.21.15.31:0/1621910576 learned_addr learned my addr 172.21.15.31:0/1621910576 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c5c89640 1 --2- 172.21.15.31:0/1621910576 >> v2:172.21.15.137:3300/0 conn(0x7fa3c00cfad0 0x7fa3c00cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c6c8b640 1 --2- 172.21.15.31:0/1621910576 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00ccc70 0x7fa3c00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 -- 172.21.15.31:0/1621910576 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00ccc70 msgr2=0x7fa3c00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 --2- 172.21.15.31:0/1621910576 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00ccc70 0x7fa3c00b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 -- 172.21.15.31:0/1621910576 >> v2:172.21.15.137:3300/0 conn(0x7fa3c00cfad0 msgr2=0x7fa3c00cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 --2- 172.21.15.31:0/1621910576 >> v2:172.21.15.137:3300/0 conn(0x7fa3c00cfad0 0x7fa3c00cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c648a640 1 -- 172.21.15.31:0/1621910576 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00b0cc0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c5c89640 1 --2- 172.21.15.31:0/1621910576 >> v2:172.21.15.137:3300/0 conn(0x7fa3c00cfad0 0x7fa3c00cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.137+0000 7fa3c6c8b640 1 --2- 172.21.15.31:0/1621910576 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00ccc70 0x7fa3c00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c648a640 1 --2- 172.21.15.31:0/1621910576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c00add00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3c015efc0 tx=0x7fa3b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ceb2606b06ff8ef server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c5488640 1 -- 172.21.15.31:0/1621910576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8031870 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c5488640 1 -- 172.21.15.31:0/1621910576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b80319d0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c5488640 1 -- 172.21.15.31:0/1621910576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8031d60 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 -- 172.21.15.31:0/1621910576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 msgr2=0x7fa3c00add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/1621910576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c00add00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3c015efc0 tx=0x7fa3b802b910 comp rx=0 tx=0).stop 2024-01-19T04:55:16.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 -- 172.21.15.31:0/1621910576 shutdown_connections 2024-01-19T04:55:16.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/1621910576 >> v2:172.21.15.137:3300/0 conn(0x7fa3c00cfad0 0x7fa3c00cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/1621910576 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00ccc70 0x7fa3c00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/1621910576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c00add00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 -- 172.21.15.31:0/1621910576 >> 172.21.15.31:0/1621910576 conn(0x7fa3c008afe0 msgr2=0x7fa3c00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 -- 172.21.15.31:0/1621910576 shutdown_connections 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.138+0000 7fa3c86ee640 1 -- 172.21.15.31:0/1621910576 wait complete. 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 Processor -- start 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 -- start start 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c00ccc70 0x7fa3c01ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00cfad0 0x7fa3c01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c015dff0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c015dd10 con 0x7fa3c00ccc70 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c86ee640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01f68f0 con 0x7fa3c00cfad0 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c5c89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c5c89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c015d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33206/0 (socket says 172.21.15.31:33206) 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.140+0000 7fa3c5c89640 1 -- 172.21.15.31:0/3804347641 learned_addr learned my addr 172.21.15.31:0/3804347641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c648a640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c00ccc70 0x7fa3c01ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c6c8b640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00cfad0 0x7fa3c01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c5c89640 1 -- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00cfad0 msgr2=0x7fa3c01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c5c89640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00cfad0 0x7fa3c01599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c5c89640 1 -- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c00ccc70 msgr2=0x7fa3c01ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c5c89640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c00ccc70 0x7fa3c01ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c5c89640 1 -- 172.21.15.31:0/3804347641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80094d0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c6c8b640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00cfad0 0x7fa3c01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c648a640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c00ccc70 0x7fa3c01ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c5c89640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c015d2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c7f0 tx=0x7fa3b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3beffd640 1 -- 172.21.15.31:0/3804347641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4007a30 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3beffd640 1 -- 172.21.15.31:0/3804347641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b4007b90 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c01f6bd0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3beffd640 1 -- 172.21.15.31:0/3804347641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400ce40 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.141+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c01ed510 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.142+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c01f6750 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.142+0000 7fa3beffd640 1 -- 172.21.15.31:0/3804347641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa3b40113f0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.143+0000 7fa3beffd640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3ac06aed0 0x7fa3ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.143+0000 7fa3beffd640 1 -- 172.21.15.31:0/3804347641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa3b4089350 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.143+0000 7fa3c648a640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3ac06aed0 0x7fa3ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.143+0000 7fa3c648a640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3ac06aed0 0x7fa3ac06d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8036000 tx=0x7fa3b803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.147+0000 7fa3beffd640 1 -- 172.21.15.31:0/3804347641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa3c01f6750 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.322+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3c01f70f0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.323+0000 7fa3beffd640 1 -- 172.21.15.31:0/3804347641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3c01f70f0 con 0x7fa3c00cf6f0 2024-01-19T04:55:16.323 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3ac06aed0 msgr2=0x7fa3ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3ac06aed0 0x7fa3ac06d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8036000 tx=0x7fa3b803c040 comp rx=0 tx=0).stop 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 msgr2=0x7fa3c015d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c015d2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c7f0 tx=0x7fa3b400cbf0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 shutdown_connections 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3ac06aed0 0x7fa3ac06d390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3c00cfad0 0x7fa3c01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c00cf6f0 0x7fa3c015d2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 --2- 172.21.15.31:0/3804347641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c00ccc70 0x7fa3c01ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 >> 172.21.15.31:0/3804347641 conn(0x7fa3c008afe0 msgr2=0x7fa3c00cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 shutdown_connections 2024-01-19T04:55:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.325+0000 7fa3c86ee640 1 -- 172.21.15.31:0/3804347641 wait complete. 2024-01-19T04:55:16.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-19T04:55:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.436+0000 7f307609b640 1 Processor -- start 2024-01-19T04:55:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.436+0000 7f307609b640 1 -- start start 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.436+0000 7f307609b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30700cca10 0x7f30700ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.436+0000 7f307609b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f307015fb80 0x7f307006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.436+0000 7f307609b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30700724e0 con 0x7f30700cff30 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.436+0000 7f307609b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30700cd330 con 0x7f30700cca10 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.436+0000 7f307609b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307006af00 con 0x7f307015fb80 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30700cca10 0x7f30700ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30700cca10 0x7f30700ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33216/0 (socket says 172.21.15.31:33216) 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306f7fe640 1 -- 172.21.15.31:0/4199954105 learned_addr learned my addr 172.21.15.31:0/4199954105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306effd640 1 --2- 172.21.15.31:0/4199954105 >> v2:172.21.15.137:3300/0 conn(0x7f307015fb80 0x7f307006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306f7fe640 1 -- 172.21.15.31:0/4199954105 >> v2:172.21.15.137:3300/0 conn(0x7f307015fb80 msgr2=0x7f307006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306f7fe640 1 --2- 172.21.15.31:0/4199954105 >> v2:172.21.15.137:3300/0 conn(0x7f307015fb80 0x7f307006a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306f7fe640 1 -- 172.21.15.31:0/4199954105 >> v1:172.21.15.137:6789/0 conn(0x7f30700cff30 legacy=0x7f30700d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306f7fe640 1 -- 172.21.15.31:0/4199954105 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307015bc10 con 0x7f30700cca10 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.437+0000 7f306effd640 1 --2- 172.21.15.31:0/4199954105 >> v2:172.21.15.137:3300/0 conn(0x7f307015fb80 0x7f307006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f306f7fe640 1 --2- 172.21.15.31:0/4199954105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30700cca10 0x7f30700ccdf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f307006f4d0 tx=0x7f306002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a773504a6ae120 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f306e7fc640 1 -- 172.21.15.31:0/4199954105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060031870 con 0x7f30700cca10 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f306e7fc640 1 -- 172.21.15.31:0/4199954105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30600319d0 con 0x7f30700cca10 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f306e7fc640 1 -- 172.21.15.31:0/4199954105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060031d60 con 0x7f30700cca10 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 -- 172.21.15.31:0/4199954105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30700cca10 msgr2=0x7f30700ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 --2- 172.21.15.31:0/4199954105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30700cca10 0x7f30700ccdf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f307006f4d0 tx=0x7f306002b910 comp rx=0 tx=0).stop 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 -- 172.21.15.31:0/4199954105 shutdown_connections 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 --2- 172.21.15.31:0/4199954105 >> v2:172.21.15.137:3300/0 conn(0x7f307015fb80 0x7f307006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 --2- 172.21.15.31:0/4199954105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30700cca10 0x7f30700ccdf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 -- 172.21.15.31:0/4199954105 >> 172.21.15.31:0/4199954105 conn(0x7f307006edc0 msgr2=0x7f307006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:16.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 -- 172.21.15.31:0/4199954105 shutdown_connections 2024-01-19T04:55:16.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.438+0000 7f307609b640 1 -- 172.21.15.31:0/4199954105 wait complete. 2024-01-19T04:55:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.439+0000 7f307609b640 1 Processor -- start 2024-01-19T04:55:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f307609b640 1 -- start start 2024-01-19T04:55:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f307609b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30700cca10 0x7f30700af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f307609b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30700cff30 0x7f30700b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f307609b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307015fb80 0x7f3070140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f307609b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700b3130 con 0x7f307015fb80 2024-01-19T04:55:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f307609b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700b3f90 con 0x7f30700cff30 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f307609b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30700b3380 con 0x7f30700cca10 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f306ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307015fb80 0x7f3070140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.440+0000 7f306ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307015fb80 0x7f3070140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33222/0 (socket says 172.21.15.31:33222) 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306ffff640 1 -- 172.21.15.31:0/1690587471 learned_addr learned my addr 172.21.15.31:0/1690587471 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306f7fe640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30700cca10 0x7f30700af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306effd640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30700cff30 0x7f30700b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306ffff640 1 -- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30700cca10 msgr2=0x7f30700af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306ffff640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30700cca10 0x7f30700af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306ffff640 1 -- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30700cff30 msgr2=0x7f30700b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306ffff640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30700cff30 0x7f30700b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306ffff640 1 -- 172.21.15.31:0/1690587471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600094d0 con 0x7f307015fb80 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f306ffff640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307015fb80 0x7f3070140e90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f306800c7b0 tx=0x7f306800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f3057fff640 1 -- 172.21.15.31:0/1690587471 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068004b60 con 0x7f307015fb80 2024-01-19T04:55:16.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f3057fff640 1 -- 172.21.15.31:0/1690587471 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3068004cc0 con 0x7f307015fb80 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30700b3660 con 0x7f307015fb80 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f3057fff640 1 -- 172.21.15.31:0/1690587471 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800aa90 con 0x7f307015fb80 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.441+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30700b3c20 con 0x7f307015fb80 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.442+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30700696e0 con 0x7f307015fb80 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.442+0000 7f3057fff640 1 -- 172.21.15.31:0/1690587471 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f306800abf0 con 0x7f307015fb80 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.443+0000 7f3057fff640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.443+0000 7f3057fff640 1 -- 172.21.15.31:0/1690587471 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3068089390 con 0x7f307015fb80 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.443+0000 7f306f7fe640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.443+0000 7f306f7fe640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3060036000 tx=0x7f306003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.447+0000 7f3057fff640 1 -- 172.21.15.31:0/1690587471 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3068054640 con 0x7f307015fb80 2024-01-19T04:55:16.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.628+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f307006d4c0 con 0x7f307015fb80 2024-01-19T04:55:16.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.628+0000 7f3057fff640 1 -- 172.21.15.31:0/1690587471 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3068057c60 con 0x7f307015fb80 2024-01-19T04:55:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:16.628 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-01-19T04:55:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:55:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-01-19T04:55:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-01-19T04:55:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:55:16.629 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 msgr2=0x7f305c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3060036000 tx=0x7f306003c040 comp rx=0 tx=0).stop 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307015fb80 msgr2=0x7f3070140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307015fb80 0x7f3070140e90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f306800c7b0 tx=0x7f306800cc70 comp rx=0 tx=0).stop 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 shutdown_connections 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307015fb80 0x7f3070140e90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30700cff30 0x7f30700b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 --2- 172.21.15.31:0/1690587471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30700cca10 0x7f30700af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 >> 172.21.15.31:0/1690587471 conn(0x7f307006edc0 msgr2=0x7f30700b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 shutdown_connections 2024-01-19T04:55:16.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.630+0000 7f307609b640 1 -- 172.21.15.31:0/1690587471 wait complete. 2024-01-19T04:55:16.640 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-19T04:55:16.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-19T04:55:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.745+0000 7f6c92008640 1 Processor -- start 2024-01-19T04:55:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c92008640 1 -- start start 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c92008640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c8c06d4c0 0x7f6c8c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c92008640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6c8c161000 0x7f6c8c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c92008640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c157f40 con 0x7f6c8c1619f0 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c92008640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c0b2c40 con 0x7f6c8c06d4c0 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c92008640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c0b2f20 con 0x7f6c8c161000 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c8affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c8c06d4c0 0x7f6c8c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c8affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c8c06d4c0 0x7f6c8c0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33226/0 (socket says 172.21.15.31:33226) 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c8b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c8c1619f0 0x7f6c8c06cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33046/0 (socket says 172.21.15.31:33046) 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.746+0000 7f6c8affd640 1 -- 172.21.15.31:0/621922780 learned_addr learned my addr 172.21.15.31:0/621922780 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:16.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.747+0000 7f6c8bfff640 1 --2- 172.21.15.31:0/621922780 >> v2:172.21.15.137:3300/0 conn(0x7f6c8c161000 0x7f6c8c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.747+0000 7f6c8affd640 1 -- 172.21.15.31:0/621922780 >> v2:172.21.15.137:3300/0 conn(0x7f6c8c161000 msgr2=0x7f6c8c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.747+0000 7f6c8affd640 1 --2- 172.21.15.31:0/621922780 >> v2:172.21.15.137:3300/0 conn(0x7f6c8c161000 0x7f6c8c0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.747+0000 7f6c8affd640 1 -- 172.21.15.31:0/621922780 >> v1:172.21.15.31:6789/0 conn(0x7f6c8c1619f0 legacy=0x7f6c8c06cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.747+0000 7f6c8affd640 1 -- 172.21.15.31:0/621922780 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c06f870 con 0x7f6c8c06d4c0 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.747+0000 7f6c8bfff640 1 --2- 172.21.15.31:0/621922780 >> v2:172.21.15.137:3300/0 conn(0x7f6c8c161000 0x7f6c8c0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.747+0000 7f6c8affd640 1 --2- 172.21.15.31:0/621922780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c06d4c0 0x7f6c8c0cd900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0d02e0 tx=0x7f6c8002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07518a66d7a1c6a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c8a7fc640 1 -- 172.21.15.31:0/621922780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80030860 con 0x7f6c8c06d4c0 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c8a7fc640 1 -- 172.21.15.31:0/621922780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c800309c0 con 0x7f6c8c06d4c0 2024-01-19T04:55:16.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c8a7fc640 1 -- 172.21.15.31:0/621922780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80030d50 con 0x7f6c8c06d4c0 2024-01-19T04:55:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 -- 172.21.15.31:0/621922780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c06d4c0 msgr2=0x7f6c8c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 --2- 172.21.15.31:0/621922780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c06d4c0 0x7f6c8c0cd900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0d02e0 tx=0x7f6c8002b9a0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 -- 172.21.15.31:0/621922780 shutdown_connections 2024-01-19T04:55:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 --2- 172.21.15.31:0/621922780 >> v2:172.21.15.137:3300/0 conn(0x7f6c8c161000 0x7f6c8c0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 --2- 172.21.15.31:0/621922780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c06d4c0 0x7f6c8c0cd900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 -- 172.21.15.31:0/621922780 >> 172.21.15.31:0/621922780 conn(0x7f6c8c066340 msgr2=0x7f6c8c066720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 -- 172.21.15.31:0/621922780 shutdown_connections 2024-01-19T04:55:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.748+0000 7f6c92008640 1 -- 172.21.15.31:0/621922780 wait complete. 2024-01-19T04:55:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.749+0000 7f6c92008640 1 Processor -- start 2024-01-19T04:55:16.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.749+0000 7f6c92008640 1 -- start start 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c92008640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c8c06d4c0 0x7f6c8c1f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c92008640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c8c161000 0x7f6c8c1f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c92008640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c1619f0 0x7f6c8c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c92008640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c158200 con 0x7f6c8c1619f0 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c92008640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1584e0 con 0x7f6c8c161000 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c92008640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1587c0 con 0x7f6c8c06d4c0 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c1619f0 0x7f6c8c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c1619f0 0x7f6c8c0c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33234/0 (socket says 172.21.15.31:33234) 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c8bfff640 1 -- 172.21.15.31:0/2782096864 learned_addr learned my addr 172.21.15.31:0/2782096864 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c8affd640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c8c161000 0x7f6c8c1f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.750+0000 7f6c8b7fe640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c8c06d4c0 0x7f6c8c1f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c8bfff640 1 -- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c8c06d4c0 msgr2=0x7f6c8c1f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c8bfff640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c8c06d4c0 0x7f6c8c1f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c8bfff640 1 -- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c8c161000 msgr2=0x7f6c8c1f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c8bfff640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c8c161000 0x7f6c8c1f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c8bfff640 1 -- 172.21.15.31:0/2782096864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c800094d0 con 0x7f6c8c1619f0 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c8affd640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c8c161000 0x7f6c8c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c8bfff640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c1619f0 0x7f6c8c0c8c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400cef0 tx=0x7f6c8400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c73fff640 1 -- 172.21.15.31:0/2782096864 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400d840 con 0x7f6c8c1619f0 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c73fff640 1 -- 172.21.15.31:0/2782096864 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8400d9c0 con 0x7f6c8c1619f0 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c73fff640 1 -- 172.21.15.31:0/2782096864 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400dd70 con 0x7f6c8c1619f0 2024-01-19T04:55:16.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c158aa0 con 0x7f6c8c1619f0 2024-01-19T04:55:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.751+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c1f66d0 con 0x7f6c8c1619f0 2024-01-19T04:55:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.752+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c158fc0 con 0x7f6c8c1619f0 2024-01-19T04:55:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.752+0000 7f6c73fff640 1 -- 172.21.15.31:0/2782096864 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6c8401e3b0 con 0x7f6c8c1619f0 2024-01-19T04:55:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.753+0000 7f6c73fff640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6c7806aed0 0x7f6c7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.753+0000 7f6c73fff640 1 -- 172.21.15.31:0/2782096864 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6c84089d40 con 0x7f6c8c1619f0 2024-01-19T04:55:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.753+0000 7f6c8b7fe640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6c7806aed0 0x7f6c7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.753+0000 7f6c8b7fe640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6c7806aed0 0x7f6c7806d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000f60 tx=0x7f6c7c007ce0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.757+0000 7f6c73fff640 1 -- 172.21.15.31:0/2782096864 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6c8c158fc0 con 0x7f6c8c1619f0 2024-01-19T04:55:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:16.938+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6c8c158fc0 con 0x7f6c8c1619f0 2024-01-19T04:55:17.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.501+0000 7f6c73fff640 1 -- 172.21.15.31:0/2782096864 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6c8c158fc0 con 0x7f6c8c1619f0 2024-01-19T04:55:17.501 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-01-19T04:55:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6c7806aed0 msgr2=0x7f6c7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6c7806aed0 0x7f6c7806d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000f60 tx=0x7f6c7c007ce0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c1619f0 msgr2=0x7f6c8c0c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c1619f0 0x7f6c8c0c8c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400cef0 tx=0x7f6c8400a710 comp rx=0 tx=0).stop 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 shutdown_connections 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6c7806aed0 0x7f6c7806d390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c8c1619f0 0x7f6c8c0c8c00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c8c161000 0x7f6c8c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 --2- 172.21.15.31:0/2782096864 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c8c06d4c0 0x7f6c8c1f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 >> 172.21.15.31:0/2782096864 conn(0x7f6c8c066340 msgr2=0x7f6c8c15fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 shutdown_connections 2024-01-19T04:55:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.504+0000 7f6c92008640 1 -- 172.21.15.31:0/2782096864 wait complete. 2024-01-19T04:55:17.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-19T04:55:17.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.630+0000 7faa4ef0e640 1 Processor -- start 2024-01-19T04:55:17.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.630+0000 7faa4ef0e640 1 -- start start 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.630+0000 7faa4ef0e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa480d11b0 0x7faa480d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.630+0000 7faa4ef0e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.630+0000 7faa4ef0e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa480cef10 con 0x7faa4815d700 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.630+0000 7faa4ef0e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa480ce300 con 0x7faa480d11b0 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.630+0000 7faa4ef0e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa480ce5e0 con 0x7faa480d1fe0 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa47fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa480d11b0 0x7faa480d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa47fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa480d11b0 0x7faa480d3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33240/0 (socket says 172.21.15.31:33240) 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa47fff640 1 -- 172.21.15.31:0/445650424 learned_addr learned my addr 172.21.15.31:0/445650424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:17.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa4d4ab640 1 --2- 172.21.15.31:0/445650424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa47fff640 1 -- 172.21.15.31:0/445650424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 msgr2=0x7faa480b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa47fff640 1 --2- 172.21.15.31:0/445650424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa47fff640 1 -- 172.21.15.31:0/445650424 >> v1:172.21.15.137:6789/0 conn(0x7faa4815d700 legacy=0x7faa480d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.631+0000 7faa47fff640 1 -- 172.21.15.31:0/445650424 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480ce8c0 con 0x7faa480d11b0 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa47fff640 1 --2- 172.21.15.31:0/445650424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa480d11b0 0x7faa480d3e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa34009870 tx=0x7faa3402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830c66973a173e9d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa477fe640 1 -- 172.21.15.31:0/445650424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3402f840 con 0x7faa480d11b0 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa477fe640 1 -- 172.21.15.31:0/445650424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3402f9a0 con 0x7faa480d11b0 2024-01-19T04:55:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa477fe640 1 -- 172.21.15.31:0/445650424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3402fd30 con 0x7faa480d11b0 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 -- 172.21.15.31:0/445650424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa480d11b0 msgr2=0x7faa480d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/445650424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa480d11b0 0x7faa480d3e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa34009870 tx=0x7faa3402ba20 comp rx=0 tx=0).stop 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 -- 172.21.15.31:0/445650424 shutdown_connections 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/445650424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/445650424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa480d11b0 0x7faa480d3e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 -- 172.21.15.31:0/445650424 >> 172.21.15.31:0/445650424 conn(0x7faa480d32f0 msgr2=0x7faa480d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 -- 172.21.15.31:0/445650424 shutdown_connections 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.632+0000 7faa4ef0e640 1 -- 172.21.15.31:0/445650424 wait complete. 2024-01-19T04:55:17.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.633+0000 7faa4ef0e640 1 Processor -- start 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.633+0000 7faa4ef0e640 1 -- start start 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4ef0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa480d11b0 0x7faa481f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4ef0e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4ef0e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4815d700 0x7faa48159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4ef0e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa48158150 con 0x7faa4815d700 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4ef0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480b5ea0 con 0x7faa480d11b0 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4ef0e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa480cb9d0 con 0x7faa480d1fe0 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4d4ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4815d700 0x7faa48159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4d4ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4815d700 0x7faa48159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33248/0 (socket says 172.21.15.31:33248) 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.634+0000 7faa4d4ab640 1 -- 172.21.15.31:0/3081309534 learned_addr learned my addr 172.21.15.31:0/3081309534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4ccaa640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa480d11b0 0x7faa481f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa47fff640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4d4ab640 1 -- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 msgr2=0x7faa480b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:17.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4d4ab640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4d4ab640 1 -- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa480d11b0 msgr2=0x7faa481f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4d4ab640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa480d11b0 0x7faa481f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4d4ab640 1 -- 172.21.15.31:0/3081309534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa340094d0 con 0x7faa4815d700 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4d4ab640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4815d700 0x7faa48159d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa40007e40 tx=0x7faa4000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa457fa640 1 -- 172.21.15.31:0/3081309534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40014620 con 0x7faa4815d700 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa457fa640 1 -- 172.21.15.31:0/3081309534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4000ee90 con 0x7faa4815d700 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa480cbcb0 con 0x7faa4815d700 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa457fa640 1 -- 172.21.15.31:0/3081309534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400149e0 con 0x7faa4815d700 2024-01-19T04:55:17.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.635+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa480cc270 con 0x7faa4815d700 2024-01-19T04:55:17.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.636+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa480b6060 con 0x7faa4815d700 2024-01-19T04:55:17.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.637+0000 7faa457fa640 1 -- 172.21.15.31:0/3081309534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7faa40014b40 con 0x7faa4815d700 2024-01-19T04:55:17.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.637+0000 7faa457fa640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa2c06aed0 0x7faa2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:17.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.637+0000 7faa457fa640 1 -- 172.21.15.31:0/3081309534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7faa4008aae0 con 0x7faa4815d700 2024-01-19T04:55:17.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.637+0000 7faa4ccaa640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa2c06aed0 0x7faa2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:17.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.638+0000 7faa4ccaa640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa2c06aed0 0x7faa2c06d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0016e0 tx=0x7faa3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:17.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.641+0000 7faa457fa640 1 -- 172.21.15.31:0/3081309534 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7faa40055d20 con 0x7faa4815d700 2024-01-19T04:55:17.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:17.822+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7faa48074900 con 0x7faa4815d700 2024-01-19T04:55:18.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.509+0000 7faa457fa640 1 -- 172.21.15.31:0/3081309534 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7faa40059340 con 0x7faa4815d700 2024-01-19T04:55:18.508 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa2c06aed0 msgr2=0x7faa2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa2c06aed0 0x7faa2c06d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0016e0 tx=0x7faa3c009040 comp rx=0 tx=0).stop 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4815d700 msgr2=0x7faa48159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4815d700 0x7faa48159d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa40007e40 tx=0x7faa4000e4b0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 shutdown_connections 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa2c06aed0 0x7faa2c06d390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4815d700 0x7faa48159d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa480d1fe0 0x7faa480b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 --2- 172.21.15.31:0/3081309534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa480d11b0 0x7faa481f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.511+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 >> 172.21.15.31:0/3081309534 conn(0x7faa480d32f0 msgr2=0x7faa480d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.512+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 shutdown_connections 2024-01-19T04:55:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.512+0000 7faa4ef0e640 1 -- 172.21.15.31:0/3081309534 wait complete. 2024-01-19T04:55:18.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-19T04:55:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.644+0000 7f7c311b0640 1 Processor -- start 2024-01-19T04:55:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.644+0000 7f7c311b0640 1 -- start start 2024-01-19T04:55:18.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.644+0000 7f7c311b0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c2c0cf6f0 0x7f7c2c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:18.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.644+0000 7f7c311b0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c0b0a70 con 0x7f7c2c0cfad0 2024-01-19T04:55:18.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.644+0000 7f7c311b0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c0b18d0 con 0x7f7c2c0b1d60 2024-01-19T04:55:18.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.644+0000 7f7c311b0640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2c0b0cc0 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c2ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c2c0cf6f0 0x7f7c2c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:18.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c2ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c2c0cf6f0 0x7f7c2c0add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33258/0 (socket says 172.21.15.31:33258) 2024-01-19T04:55:18.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c2ad9d640 1 -- 172.21.15.31:0/1718576183 learned_addr learned my addr 172.21.15.31:0/1718576183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c23fff640 1 --1- 172.21.15.31:0/1718576183 >> v1:172.21.15.31:6789/0 conn(0x7f7c2c0cfad0 0x7f7c2c0cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33050/0 (socket says 172.21.15.31:33050) 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c2ad9d640 1 -- 172.21.15.31:0/1718576183 >> v1:172.21.15.137:6789/0 conn(0x7f7c2c0b1d60 legacy=0x7f7c2c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c2ad9d640 1 -- 172.21.15.31:0/1718576183 >> v1:172.21.15.31:6789/0 conn(0x7f7c2c0cfad0 legacy=0x7f7c2c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c2a59c640 1 -- 172.21.15.31:0/1718576183 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857704430 0 0) 0x7f7c2c0b0a70 con 0x7f7c2c0cfad0 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.645+0000 7f7c2ad9d640 1 -- 172.21.15.31:0/1718576183 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0b0fa0 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c2ad9d640 1 --2- 172.21.15.31:0/1718576183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c0add00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009870 tx=0x7f7c1c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23f352d5e7f7246 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c2a59c640 1 -- 172.21.15.31:0/1718576183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02f840 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c2a59c640 1 -- 172.21.15.31:0/1718576183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c02f9a0 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c2a59c640 1 -- 172.21.15.31:0/1718576183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02fd30 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c311b0640 1 -- 172.21.15.31:0/1718576183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 msgr2=0x7f7c2c0add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c311b0640 1 --2- 172.21.15.31:0/1718576183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c0add00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009870 tx=0x7f7c1c02ba20 comp rx=0 tx=0).stop 2024-01-19T04:55:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c311b0640 1 -- 172.21.15.31:0/1718576183 shutdown_connections 2024-01-19T04:55:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c311b0640 1 --2- 172.21.15.31:0/1718576183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c0add00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c311b0640 1 -- 172.21.15.31:0/1718576183 >> 172.21.15.31:0/1718576183 conn(0x7f7c2c08afe0 msgr2=0x7f7c2c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c311b0640 1 -- 172.21.15.31:0/1718576183 shutdown_connections 2024-01-19T04:55:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.646+0000 7f7c311b0640 1 -- 172.21.15.31:0/1718576183 wait complete. 2024-01-19T04:55:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 Processor -- start 2024-01-19T04:55:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 -- start start 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c2c0b1d60 0x7f7c2c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c1f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c2c0cfad0 0x7f7c2c0c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c14c650 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0c68a0 con 0x7f7c2c0cfad0 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c311b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0c5c90 con 0x7f7c2c0b1d60 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c1f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c1f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33268/0 (socket says 172.21.15.31:33268) 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.648+0000 7f7c23fff640 1 -- 172.21.15.31:0/4233953435 learned_addr learned my addr 172.21.15.31:0/4233953435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:18.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c2b59e640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c2c0cfad0 0x7f7c2c0c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c2ad9d640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c2c0b1d60 0x7f7c2c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c23fff640 1 -- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c2c0b1d60 msgr2=0x7f7c2c14c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c23fff640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c2c0b1d60 0x7f7c2c14c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c23fff640 1 -- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c2c0cfad0 msgr2=0x7f7c2c0c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c23fff640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c2c0cfad0 0x7f7c2c0c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c23fff640 1 -- 172.21.15.31:0/4233953435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0094d0 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c23fff640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c1f3590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c3f0 tx=0x7f7c1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c237fe640 1 -- 172.21.15.31:0/4233953435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c140074e0 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c237fe640 1 -- 172.21.15.31:0/4233953435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c14007640 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c0c5f70 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c237fe640 1 -- 172.21.15.31:0/4233953435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14004a00 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.649+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c0c6530 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.650+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c074900 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.651+0000 7f7c237fe640 1 -- 172.21.15.31:0/4233953435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7c14004b60 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.651+0000 7f7c237fe640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c0c06aed0 0x7f7c0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.651+0000 7f7c237fe640 1 -- 172.21.15.31:0/4233953435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7c140891f0 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.651+0000 7f7c2ad9d640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c0c06aed0 0x7f7c0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.652+0000 7f7c2ad9d640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c0c06aed0 0x7f7c0c06d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c036000 tx=0x7f7c1c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.656+0000 7f7c237fe640 1 -- 172.21.15.31:0/4233953435 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7c14054260 con 0x7f7c2c0cf6f0 2024-01-19T04:55:18.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:18.875+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7c2c0cf4d0 con 0x7f7c2c0cf6f0 2024-01-19T04:55:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.512+0000 7f7c237fe640 1 -- 172.21.15.31:0/4233953435 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7c14057880 con 0x7f7c2c0cf6f0 2024-01-19T04:55:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-01-19T04:55:19.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c0c06aed0 msgr2=0x7f7c0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c0c06aed0 0x7f7c0c06d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c036000 tx=0x7f7c1c008030 comp rx=0 tx=0).stop 2024-01-19T04:55:19.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 msgr2=0x7f7c2c1f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c1f3590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c3f0 tx=0x7f7c1400c8b0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 shutdown_connections 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c0c06aed0 0x7f7c0c06d390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c2c0cfad0 0x7f7c2c0c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c2c0cf6f0 0x7f7c2c1f3590 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 --2- 172.21.15.31:0/4233953435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c2c0b1d60 0x7f7c2c14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 >> 172.21.15.31:0/4233953435 conn(0x7f7c2c08afe0 msgr2=0x7f7c2c0afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.515+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 shutdown_connections 2024-01-19T04:55:19.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.516+0000 7f7c311b0640 1 -- 172.21.15.31:0/4233953435 wait complete. 2024-01-19T04:55:19.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.639+0000 7f1c99858640 1 Processor -- start 2024-01-19T04:55:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.639+0000 7f1c99858640 1 -- start start 2024-01-19T04:55:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c99858640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c94160a10 0x7f1c940b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c99858640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c94160df0 con 0x7f1c940b4d00 2024-01-19T04:55:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c99858640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940d1db0 con 0x7f1c940d2e50 2024-01-19T04:55:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c99858640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c940d2090 con 0x7f1c94160a10 2024-01-19T04:55:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c92ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c94160a10 0x7f1c940b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c937fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1c940b4d00 0x7f1c940b62f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56764/0 (socket says 172.21.15.31:56764) 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c92ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c94160a10 0x7f1c940b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35818/0 (socket says 172.21.15.31:35818) 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c937fe640 1 -- 172.21.15.31:0/2015527021 learned_addr learned my addr 172.21.15.31:0/2015527021 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c92ffd640 1 -- 172.21.15.31:0/2015527021 >> v1:172.21.15.137:6789/0 conn(0x7f1c940d2e50 legacy=0x7f1c940d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c92ffd640 1 -- 172.21.15.31:0/2015527021 >> v1:172.21.15.31:6789/0 conn(0x7f1c940b4d00 legacy=0x7f1c940b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c91ffb640 1 -- 172.21.15.31:0/2015527021 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163377613 0 0) 0x7f1c94160df0 con 0x7f1c940b4d00 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.640+0000 7f1c92ffd640 1 -- 172.21.15.31:0/2015527021 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940d2370 con 0x7f1c94160a10 2024-01-19T04:55:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.641+0000 7f1c92ffd640 1 --2- 172.21.15.31:0/2015527021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c94160a10 0x7f1c940b8ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0097a0 tx=0x7f1c8c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3410d9ffaca8b7e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.641+0000 7f1c91ffb640 1 -- 172.21.15.31:0/2015527021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c030610 con 0x7f1c94160a10 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.641+0000 7f1c91ffb640 1 -- 172.21.15.31:0/2015527021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c030770 con 0x7f1c94160a10 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.641+0000 7f1c91ffb640 1 -- 172.21.15.31:0/2015527021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c030b00 con 0x7f1c94160a10 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.642+0000 7f1c99858640 1 -- 172.21.15.31:0/2015527021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c94160a10 msgr2=0x7f1c940b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.642+0000 7f1c99858640 1 --2- 172.21.15.31:0/2015527021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c94160a10 0x7f1c940b8ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0097a0 tx=0x7f1c8c02b960 comp rx=0 tx=0).stop 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.642+0000 7f1c99858640 1 -- 172.21.15.31:0/2015527021 shutdown_connections 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.642+0000 7f1c99858640 1 --2- 172.21.15.31:0/2015527021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c94160a10 0x7f1c940b8ea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.642+0000 7f1c99858640 1 -- 172.21.15.31:0/2015527021 >> 172.21.15.31:0/2015527021 conn(0x7f1c94072310 msgr2=0x7f1c940cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.642+0000 7f1c99858640 1 -- 172.21.15.31:0/2015527021 shutdown_connections 2024-01-19T04:55:19.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.642+0000 7f1c99858640 1 -- 172.21.15.31:0/2015527021 wait complete. 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.643+0000 7f1c99858640 1 Processor -- start 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.643+0000 7f1c99858640 1 -- start start 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c99858640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1c940b4d00 0x7f1c94156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c99858640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c940d2e50 0x7f1c940c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c99858640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c94160a10 0x7f1c94157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c99858640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94156900 con 0x7f1c940d2e50 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c99858640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94157260 con 0x7f1c94160a10 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c99858640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94155c80 con 0x7f1c940b4d00 2024-01-19T04:55:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c940d2e50 0x7f1c940c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c940d2e50 0x7f1c940c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35824/0 (socket says 172.21.15.31:35824) 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 -- 172.21.15.31:0/3402236567 learned_addr learned my addr 172.21.15.31:0/3402236567 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c92ffd640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1c940b4d00 0x7f1c94156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 -- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1c940b4d00 msgr2=0x7f1c94156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c937fe640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c94160a10 0x7f1c94157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1c940b4d00 0x7f1c94156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 -- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c94160a10 msgr2=0x7f1c94157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c94160a10 0x7f1c94157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 -- 172.21.15.31:0/3402236567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c009400 con 0x7f1c940d2e50 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c937fe640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c94160a10 0x7f1c94157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c92ffd640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1c940b4d00 0x7f1c94156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.644+0000 7f1c927fc640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c940d2e50 0x7f1c940c8e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c400 tx=0x7f1c7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.645+0000 7f1c8bfff640 1 -- 172.21.15.31:0/3402236567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0052d0 con 0x7f1c940d2e50 2024-01-19T04:55:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.645+0000 7f1c8bfff640 1 -- 172.21.15.31:0/3402236567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c005430 con 0x7f1c940d2e50 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.645+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c94155f60 con 0x7f1c940d2e50 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.645+0000 7f1c8bfff640 1 -- 172.21.15.31:0/3402236567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c003830 con 0x7f1c940d2e50 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.645+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c94156520 con 0x7f1c940d2e50 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.646+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c94156a40 con 0x7f1c940d2e50 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.646+0000 7f1c8bfff640 1 -- 172.21.15.31:0/3402236567 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1c7c003990 con 0x7f1c940d2e50 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.646+0000 7f1c8bfff640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1c7406aed0 0x7f1c7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.646+0000 7f1c8bfff640 1 -- 172.21.15.31:0/3402236567 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c7c088c90 con 0x7f1c940d2e50 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.647+0000 7f1c92ffd640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1c7406aed0 0x7f1c7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.647+0000 7f1c92ffd640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1c7406aed0 0x7f1c7406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c036000 tx=0x7f1c8c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.651+0000 7f1c8bfff640 1 -- 172.21.15.31:0/3402236567 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1c94156a40 con 0x7f1c940d2e50 2024-01-19T04:55:19.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.829+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c94156a40 con 0x7f1c940d2e50 2024-01-19T04:55:19.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.830+0000 7f1c8bfff640 1 -- 172.21.15.31:0/3402236567 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16483 (secure 0 0 0) 0x7f1c94156a40 con 0x7f1c940d2e50 2024-01-19T04:55:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:19.830 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:18.876811+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:19.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.832+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1c7406aed0 msgr2=0x7f1c7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1c7406aed0 0x7f1c7406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c036000 tx=0x7f1c8c03b040 comp rx=0 tx=0).stop 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c940d2e50 msgr2=0x7f1c940c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c940d2e50 0x7f1c940c8e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c400 tx=0x7f1c7c00c8c0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 shutdown_connections 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1c7406aed0 0x7f1c7406d390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c94160a10 0x7f1c94157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c940d2e50 0x7f1c940c8e00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 --2- 172.21.15.31:0/3402236567 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1c940b4d00 0x7f1c94156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 >> 172.21.15.31:0/3402236567 conn(0x7f1c94072310 msgr2=0x7f1c9415f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 shutdown_connections 2024-01-19T04:55:19.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.833+0000 7f1c99858640 1 -- 172.21.15.31:0/3402236567 wait complete. 2024-01-19T04:55:19.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.944+0000 7fe85cf82640 1 Processor -- start 2024-01-19T04:55:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.944+0000 7fe85cf82640 1 -- start start 2024-01-19T04:55:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe85cf82640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe8580cfd10 0x7fe8580d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe85cf82640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580d06b0 con 0x7fe8580d0ef0 2024-01-19T04:55:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe85cf82640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580d1b90 con 0x7fe858160bb0 2024-01-19T04:55:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe85cf82640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8580d1e70 con 0x7fe8580cfd10 2024-01-19T04:55:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe856d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe8580d0ef0 0x7fe8580d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56774/0 (socket says 172.21.15.31:56774) 2024-01-19T04:55:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe856d9d640 1 -- 172.21.15.31:0/3912441447 learned_addr learned my addr 172.21.15.31:0/3912441447 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe855d9b640 1 --2- 172.21.15.31:0/3912441447 >> v2:172.21.15.137:3300/0 conn(0x7fe8580cfd10 0x7fe8580d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037222581 0 0) 0x7fe8580d06b0 con 0x7fe8580d0ef0 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe844003520 con 0x7fe8580d0ef0 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.945+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1792570657 0 0) 0x7fe844003520 con 0x7fe8580d0ef0 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 >> v2:172.21.15.137:3300/0 conn(0x7fe8580cfd10 msgr2=0x7fe8580d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 --2- 172.21.15.31:0/3912441447 >> v2:172.21.15.137:3300/0 conn(0x7fe8580cfd10 0x7fe8580d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 >> v1:172.21.15.137:6789/0 conn(0x7fe858160bb0 legacy=0x7fe8580b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580d2150 con 0x7fe8580d0ef0 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fe8480019f0 con 0x7fe8580d0ef0 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 >> v1:172.21.15.31:6789/0 conn(0x7fe8580d0ef0 legacy=0x7fe8580d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe844003970 0x7fe844003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8440047c0 0x7fe844004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe844005180 0x7fe84400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580d1e70 con 0x7fe844003970 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84400c020 con 0x7fe8440047c0 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84400c1e0 con 0x7fe844005180 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85659c640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe844003970 0x7fe844003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe855d9b640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8440047c0 0x7fe844004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe856d9d640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe844005180 0x7fe84400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85659c640 1 -- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe844005180 msgr2=0x7fe84400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85659c640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe844005180 0x7fe84400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85659c640 1 -- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8440047c0 msgr2=0x7fe844004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85659c640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8440047c0 0x7fe844004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe85659c640 1 -- 172.21.15.31:0/3912441447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844003520 con 0x7fe844003970 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe856d9d640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe844005180 0x7fe84400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.946+0000 7fe856d9d640 1 -- 172.21.15.31:0/3912441447 reap_dead start 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85659c640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe844003970 0x7fe844003d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe850001750 tx=0x7fe8440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afdd9d3b3d9d737 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500052d0 con 0x7fe844003970 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85002e050 con 0x7fe844003970 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85559a640 1 -- 172.21.15.31:0/3912441447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850003830 con 0x7fe844003970 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85cf82640 1 -- 172.21.15.31:0/3912441447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe844003970 msgr2=0x7fe844003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85cf82640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe844003970 0x7fe844003d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe850001750 tx=0x7fe8440057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85cf82640 1 -- 172.21.15.31:0/3912441447 shutdown_connections 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85cf82640 1 --2- 172.21.15.31:0/3912441447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe844003970 0x7fe844003d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.947+0000 7fe85cf82640 1 -- 172.21.15.31:0/3912441447 >> 172.21.15.31:0/3912441447 conn(0x7fe85809c600 msgr2=0x7fe8580d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.948+0000 7fe85cf82640 1 -- 172.21.15.31:0/3912441447 shutdown_connections 2024-01-19T04:55:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.948+0000 7fe85cf82640 1 -- 172.21.15.31:0/3912441447 wait complete. 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.949+0000 7fe85cf82640 1 Processor -- start 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.949+0000 7fe85cf82640 1 -- start start 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe844003970 0x7fe858151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580b7020 0x7fe8580b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe85815f130 0x7fe85815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85815faf0 con 0x7fe8580b7020 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581f3380 con 0x7fe844003970 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580cd280 con 0x7fe85815f130 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580b7020 0x7fe8580b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580b7020 0x7fe8580b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35836/0 (socket says 172.21.15.31:35836) 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 -- 172.21.15.31:0/2329578331 learned_addr learned my addr 172.21.15.31:0/2329578331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85659c640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe844003970 0x7fe858151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe856d9d640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe85815f130 0x7fe85815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 -- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe85815f130 msgr2=0x7fe85815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe85815f130 0x7fe85815f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 -- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe844003970 msgr2=0x7fe858151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe844003970 0x7fe858151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 -- 172.21.15.31:0/2329578331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858151c70 con 0x7fe8580b7020 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe856d9d640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe85815f130 0x7fe85815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85659c640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe844003970 0x7fe858151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe855d9b640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580b7020 0x7fe8580b7460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe840011aa0 tx=0x7fe840011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe84effd640 1 -- 172.21.15.31:0/2329578331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840009680 con 0x7fe8580b7020 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe84effd640 1 -- 172.21.15.31:0/2329578331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84000fb20 con 0x7fe8580b7020 2024-01-19T04:55:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8580cc700 con 0x7fe8580b7020 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe84effd640 1 -- 172.21.15.31:0/2329578331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000feb0 con 0x7fe8580b7020 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.950+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580cccc0 con 0x7fe8580b7020 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.951+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8580696e0 con 0x7fe8580b7020 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.952+0000 7fe84effd640 1 -- 172.21.15.31:0/2329578331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe840014440 con 0x7fe8580b7020 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.952+0000 7fe84effd640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe83806af00 0x7fe83806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.952+0000 7fe84effd640 1 -- 172.21.15.31:0/2329578331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe840089d00 con 0x7fe8580b7020 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.952+0000 7fe85659c640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe83806af00 0x7fe83806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.953+0000 7fe85659c640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe83806af00 0x7fe83806d3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe850033000 tx=0x7fe850009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:19.956+0000 7fe84effd640 1 -- 172.21.15.31:0/2329578331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe840054c60 con 0x7fe8580b7020 2024-01-19T04:55:20.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.134+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe85808d3b0 con 0x7fe8580b7020 2024-01-19T04:55:20.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.135+0000 7fe84effd640 1 -- 172.21.15.31:0/2329578331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16483 (secure 0 0 0) 0x7fe840081e60 con 0x7fe8580b7020 2024-01-19T04:55:20.135 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:20.135 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:18.876811+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe83806af00 msgr2=0x7fe83806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe83806af00 0x7fe83806d3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe850033000 tx=0x7fe850009030 comp rx=0 tx=0).stop 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580b7020 msgr2=0x7fe8580b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580b7020 0x7fe8580b7460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe840011aa0 tx=0x7fe840011f60 comp rx=0 tx=0).stop 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 shutdown_connections 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe83806af00 0x7fe83806d3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe85815f130 0x7fe85815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580b7020 0x7fe8580b7460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 --2- 172.21.15.31:0/2329578331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe844003970 0x7fe858151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 >> 172.21.15.31:0/2329578331 conn(0x7fe85809c600 msgr2=0x7fe8580cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 shutdown_connections 2024-01-19T04:55:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.137+0000 7fe85cf82640 1 -- 172.21.15.31:0/2329578331 wait complete. 2024-01-19T04:55:20.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-19T04:55:20.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.252+0000 7f5161951640 1 Processor -- start 2024-01-19T04:55:20.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.252+0000 7f5161951640 1 -- start start 2024-01-19T04:55:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f5161951640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f515c15d720 0x7f515c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f5161951640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d3ea0 0x7f515c0d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f5161951640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f515c0d1a50 0x7f515c0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f5161951640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515c0d1f00 con 0x7f515c0d1a50 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f5161951640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515c0cf160 con 0x7f515c15d720 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f5161951640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0ce8c0 con 0x7f515c0d3ea0 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f515c0d1a50 0x7f515c0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f515c0d1a50 0x7f515c0cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35852/0 (socket says 172.21.15.31:35852) 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 -- 172.21.15.31:0/1862340942 learned_addr learned my addr 172.21.15.31:0/1862340942 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515affd640 1 --2- 172.21.15.31:0/1862340942 >> v2:172.21.15.137:3300/0 conn(0x7f515c15d720 0x7f515c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515a7fc640 1 --2- 172.21.15.31:0/1862340942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d3ea0 0x7f515c0d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 -- 172.21.15.31:0/1862340942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d3ea0 msgr2=0x7f515c0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 --2- 172.21.15.31:0/1862340942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d3ea0 0x7f515c0d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 -- 172.21.15.31:0/1862340942 >> v2:172.21.15.137:3300/0 conn(0x7f515c15d720 msgr2=0x7f515c0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 --2- 172.21.15.31:0/1862340942 >> v2:172.21.15.137:3300/0 conn(0x7f515c15d720 0x7f515c0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515b7fe640 1 -- 172.21.15.31:0/1862340942 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0ceba0 con 0x7f515c0d1a50 2024-01-19T04:55:20.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.253+0000 7f515affd640 1 --2- 172.21.15.31:0/1862340942 >> v2:172.21.15.137:3300/0 conn(0x7f515c15d720 0x7f515c0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.254+0000 7f515b7fe640 1 --2- 172.21.15.31:0/1862340942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c0d1a50 0x7f515c0cfc70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f515c067310 tx=0x7f514c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b112b6a3c2e17a3b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.254+0000 7f5159ffb640 1 -- 172.21.15.31:0/1862340942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c02c950 con 0x7f515c0d1a50 2024-01-19T04:55:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.254+0000 7f5159ffb640 1 -- 172.21.15.31:0/1862340942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c02cab0 con 0x7f515c0d1a50 2024-01-19T04:55:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.254+0000 7f5159ffb640 1 -- 172.21.15.31:0/1862340942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c02ce40 con 0x7f515c0d1a50 2024-01-19T04:55:20.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 -- 172.21.15.31:0/1862340942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c0d1a50 msgr2=0x7f515c0cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 --2- 172.21.15.31:0/1862340942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c0d1a50 0x7f515c0cfc70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f515c067310 tx=0x7f514c02bdb0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 -- 172.21.15.31:0/1862340942 shutdown_connections 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 --2- 172.21.15.31:0/1862340942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c0d1a50 0x7f515c0cfc70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 --2- 172.21.15.31:0/1862340942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d3ea0 0x7f515c0d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 --2- 172.21.15.31:0/1862340942 >> v2:172.21.15.137:3300/0 conn(0x7f515c15d720 0x7f515c0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 -- 172.21.15.31:0/1862340942 >> 172.21.15.31:0/1862340942 conn(0x7f515c0d3310 msgr2=0x7f515c0d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 -- 172.21.15.31:0/1862340942 shutdown_connections 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.255+0000 7f5161951640 1 -- 172.21.15.31:0/1862340942 wait complete. 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.256+0000 7f5161951640 1 Processor -- start 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.256+0000 7f5161951640 1 -- start start 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5161951640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d1a50 0x7f515c15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5161951640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f515c0d3ea0 0x7f515c0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5161951640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c15d720 0x7f515c146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5161951640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c15e4a0 con 0x7f515c15d720 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5161951640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0c9440 con 0x7f515c0d3ea0 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5161951640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c15ae50 con 0x7f515c0d1a50 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c15d720 0x7f515c146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c15d720 0x7f515c146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35868/0 (socket says 172.21.15.31:35868) 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 -- 172.21.15.31:0/2063427853 learned_addr learned my addr 172.21.15.31:0/2063427853 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515affd640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d1a50 0x7f515c15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515a7fc640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f515c0d3ea0 0x7f515c0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 -- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d1a50 msgr2=0x7f515c15b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d1a50 0x7f515c15b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 -- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f515c0d3ea0 msgr2=0x7f515c0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f515c0d3ea0 0x7f515c0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 -- 172.21.15.31:0/2063427853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c009b30 con 0x7f515c15d720 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515a7fc640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f515c0d3ea0 0x7f515c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515affd640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d1a50 0x7f515c15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f515b7fe640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c15d720 0x7f515c146500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f514c009f40 tx=0x7f514c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:20.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5153fff640 1 -- 172.21.15.31:0/2063427853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c009510 con 0x7f515c15d720 2024-01-19T04:55:20.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5153fff640 1 -- 172.21.15.31:0/2063427853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c03e450 con 0x7f515c15d720 2024-01-19T04:55:20.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5153fff640 1 -- 172.21.15.31:0/2063427853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c03e800 con 0x7f515c15d720 2024-01-19T04:55:20.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.257+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c15a5a0 con 0x7f515c15d720 2024-01-19T04:55:20.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.258+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c15e1c0 con 0x7f515c15d720 2024-01-19T04:55:20.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.258+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c074900 con 0x7f515c15d720 2024-01-19T04:55:20.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.259+0000 7f5153fff640 1 -- 172.21.15.31:0/2063427853 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f514c03e960 con 0x7f515c15d720 2024-01-19T04:55:20.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.259+0000 7f5153fff640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f513c06aed0 0x7f513c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.259+0000 7f5153fff640 1 -- 172.21.15.31:0/2063427853 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f514c0ac430 con 0x7f515c15d720 2024-01-19T04:55:20.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.260+0000 7f515affd640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f513c06aed0 0x7f513c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.260+0000 7f515affd640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f513c06aed0 0x7f513c06d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f515c1f65f0 tx=0x7f5154008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:20.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.264+0000 7f5153fff640 1 -- 172.21.15.31:0/2063427853 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f514c077390 con 0x7f515c15d720 2024-01-19T04:55:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.478+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f515c15af90 con 0x7f515c15d720 2024-01-19T04:55:20.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.523+0000 7f5153fff640 1 -- 172.21.15.31:0/2063427853 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f514c07a9b0 con 0x7f515c15d720 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f513c06aed0 msgr2=0x7f513c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f513c06aed0 0x7f513c06d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f515c1f65f0 tx=0x7f5154008040 comp rx=0 tx=0).stop 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c15d720 msgr2=0x7f515c146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c15d720 0x7f515c146500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f514c009f40 tx=0x7f514c004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 shutdown_connections 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f513c06aed0 0x7f513c06d390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515c15d720 0x7f515c146500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f515c0d3ea0 0x7f515c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 --2- 172.21.15.31:0/2063427853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f515c0d1a50 0x7f515c15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 >> 172.21.15.31:0/2063427853 conn(0x7f515c0d3310 msgr2=0x7f515c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 shutdown_connections 2024-01-19T04:55:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.525+0000 7f5161951640 1 -- 172.21.15.31:0/2063427853 wait complete. 2024-01-19T04:55:20.536 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-19T04:55:20.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-19T04:55:20.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.640+0000 7f2044696640 1 Processor -- start 2024-01-19T04:55:20.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.640+0000 7f2044696640 1 -- start start 2024-01-19T04:55:20.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.641+0000 7f2044696640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f203c15d700 0x7f203c0d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.641+0000 7f2044696640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c0cef10 con 0x7f203c0d11b0 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.641+0000 7f2044696640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c0ce300 con 0x7f203c0d1fe0 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.641+0000 7f2044696640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0ce5e0 con 0x7f203c15d700 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.641+0000 7f2041c31640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f203c0d11b0 0x7f203c0d18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56790/0 (socket says 172.21.15.31:56790) 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.641+0000 7f2041c31640 1 -- 172.21.15.31:0/54942967 learned_addr learned my addr 172.21.15.31:0/54942967 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969892191 0 0) 0x7f203c0cef10 con 0x7f203c0d11b0 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2028003520 con 0x7f203c0d11b0 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2042432640 1 --2- 172.21.15.31:0/54942967 >> v2:172.21.15.137:3300/0 conn(0x7f203c15d700 0x7f203c0d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679035826 0 0) 0x7f2028003520 con 0x7f203c0d11b0 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 >> v2:172.21.15.137:3300/0 conn(0x7f203c15d700 msgr2=0x7f203c0d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 --2- 172.21.15.31:0/54942967 >> v2:172.21.15.137:3300/0 conn(0x7f203c15d700 0x7f203c0d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 >> v1:172.21.15.137:6789/0 conn(0x7f203c0d1fe0 legacy=0x7f203c0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0ce8c0 con 0x7f203c0d11b0 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f202c001670 con 0x7f203c0d11b0 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 >> v1:172.21.15.31:6789/0 conn(0x7f203c0d11b0 legacy=0x7f203c0d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028003970 0x7f2028003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20280047c0 0x7f2028004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2028005180 0x7f202800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0ce5e0 con 0x7f20280047c0 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202800c030 con 0x7f2028003970 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202800c1f0 con 0x7f2028005180 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041c31640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20280047c0 0x7f2028004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2042432640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028003970 0x7f2028003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2042c33640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2028005180 0x7f202800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041c31640 1 -- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2028005180 msgr2=0x7f202800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041c31640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2028005180 0x7f202800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041c31640 1 -- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028003970 msgr2=0x7f2028003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:20.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041c31640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028003970 0x7f2028003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2041c31640 1 -- 172.21.15.31:0/54942967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028003520 con 0x7f20280047c0 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.642+0000 7f2042c33640 1 -- 172.21.15.31:0/54942967 reap_dead start 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.643+0000 7f2041c31640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20280047c0 0x7f2028004c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f202c001050 tx=0x7f20280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eff34e52dd72cb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.643+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c005ad0 con 0x7f20280047c0 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.643+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c005c30 con 0x7f20280047c0 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.643+0000 7f2041430640 1 -- 172.21.15.31:0/54942967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c02cad0 con 0x7f20280047c0 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.644+0000 7f2044696640 1 -- 172.21.15.31:0/54942967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20280047c0 msgr2=0x7f2028004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.644+0000 7f2044696640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20280047c0 0x7f2028004c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f202c001050 tx=0x7f20280057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.644+0000 7f2044696640 1 -- 172.21.15.31:0/54942967 shutdown_connections 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.644+0000 7f2044696640 1 --2- 172.21.15.31:0/54942967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20280047c0 0x7f2028004c40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.644+0000 7f2044696640 1 -- 172.21.15.31:0/54942967 >> 172.21.15.31:0/54942967 conn(0x7f203c0d32f0 msgr2=0x7f203c0d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.644+0000 7f2044696640 1 -- 172.21.15.31:0/54942967 shutdown_connections 2024-01-19T04:55:20.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.644+0000 7f2044696640 1 -- 172.21.15.31:0/54942967 wait complete. 2024-01-19T04:55:20.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.645+0000 7f2044696640 1 Processor -- start 2024-01-19T04:55:20.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.645+0000 7f2044696640 1 -- start start 2024-01-19T04:55:20.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2044696640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20280047c0 0x7f203c0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2044696640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f203c0c9960 0x7f203c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2044696640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f203c0c7300 0x7f203c160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2044696640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c15f510 con 0x7f203c0c9960 2024-01-19T04:55:20.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2044696640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c15fdb0 con 0x7f203c0c7300 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2044696640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0cc960 con 0x7f20280047c0 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f203c0c9960 0x7f203c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f203c0c9960 0x7f203c0c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35894/0 (socket says 172.21.15.31:35894) 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 -- 172.21.15.31:0/1559787940 learned_addr learned my addr 172.21.15.31:0/1559787940 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2042c33640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f203c0c7300 0x7f203c160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2042432640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20280047c0 0x7f203c0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 -- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20280047c0 msgr2=0x7f203c0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20280047c0 0x7f203c0cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 -- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f203c0c7300 msgr2=0x7f203c160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f203c0c7300 0x7f203c160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 -- 172.21.15.31:0/1559787940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c15f760 con 0x7f203c0c9960 2024-01-19T04:55:20.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2041c31640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f203c0c9960 0x7f203c0c6dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f202c001050 tx=0x7f202c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:20.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2032ffd640 1 -- 172.21.15.31:0/1559787940 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c03c070 con 0x7f203c0c9960 2024-01-19T04:55:20.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2032ffd640 1 -- 172.21.15.31:0/1559787940 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c02cb60 con 0x7f203c0c9960 2024-01-19T04:55:20.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c15fa40 con 0x7f203c0c9960 2024-01-19T04:55:20.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.646+0000 7f2032ffd640 1 -- 172.21.15.31:0/1559787940 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c007070 con 0x7f203c0c9960 2024-01-19T04:55:20.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.647+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c0cc710 con 0x7f203c0c9960 2024-01-19T04:55:20.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.647+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c074900 con 0x7f203c0c9960 2024-01-19T04:55:20.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.648+0000 7f2032ffd640 1 -- 172.21.15.31:0/1559787940 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f202c032070 con 0x7f203c0c9960 2024-01-19T04:55:20.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.648+0000 7f2032ffd640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f202006aed0 0x7f202006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:20.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.648+0000 7f2032ffd640 1 -- 172.21.15.31:0/1559787940 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f202c0aa590 con 0x7f203c0c9960 2024-01-19T04:55:20.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.649+0000 7f2042432640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f202006aed0 0x7f202006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:20.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.649+0000 7f2042432640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f202006aed0 0x7f202006d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f203c0a1220 tx=0x7f2034008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:20.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.653+0000 7f2032ffd640 1 -- 172.21.15.31:0/1559787940 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f202c0753c0 con 0x7f203c0c9960 2024-01-19T04:55:20.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:20.869+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f203c0d16b0 con 0x7f203c0c9960 2024-01-19T04:55:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.524+0000 7f2032ffd640 1 -- 172.21.15.31:0/1559787940 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f202c0789e0 con 0x7f203c0c9960 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f202006aed0 msgr2=0x7f202006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f202006aed0 0x7f202006d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f203c0a1220 tx=0x7f2034008040 comp rx=0 tx=0).stop 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f203c0c9960 msgr2=0x7f203c0c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f203c0c9960 0x7f203c0c6dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f202c001050 tx=0x7f202c000fe0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 shutdown_connections 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f202006aed0 0x7f202006d390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f203c0c7300 0x7f203c160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f203c0c9960 0x7f203c0c6dc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 --2- 172.21.15.31:0/1559787940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20280047c0 0x7f203c0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 >> 172.21.15.31:0/1559787940 conn(0x7f203c0d32f0 msgr2=0x7f203c0b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 shutdown_connections 2024-01-19T04:55:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.527+0000 7f2044696640 1 -- 172.21.15.31:0/1559787940 wait complete. 2024-01-19T04:55:21.540 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-19T04:55:21.540 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:55:21.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e70914640 1 Processor -- start 2024-01-19T04:55:21.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e70914640 1 -- start start 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e70914640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e6c0cfd10 0x7f5e6c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e70914640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c0d06b0 con 0x7f5e6c160bb0 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e70914640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c0d1b90 con 0x7f5e6c0d0ef0 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e70914640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6c0d1e70 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e61d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e6c0cfd10 0x7f5e6c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e61d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e6c0cfd10 0x7f5e6c0d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35900/0 (socket says 172.21.15.31:35900) 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e61d9b640 1 -- 172.21.15.31:0/284508072 learned_addr learned my addr 172.21.15.31:0/284508072 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:21.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.648+0000 7f5e6a59c640 1 --1- 172.21.15.31:0/284508072 >> v1:172.21.15.31:6789/0 conn(0x7f5e6c160bb0 0x7f5e6c0b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56796/0 (socket says 172.21.15.31:56796) 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.649+0000 7f5e61d9b640 1 -- 172.21.15.31:0/284508072 >> v1:172.21.15.137:6789/0 conn(0x7f5e6c0d0ef0 legacy=0x7f5e6c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.649+0000 7f5e61d9b640 1 -- 172.21.15.31:0/284508072 >> v1:172.21.15.31:6789/0 conn(0x7f5e6c160bb0 legacy=0x7f5e6c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.649+0000 7f5e69d9b640 1 -- 172.21.15.31:0/284508072 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785121303 0 0) 0x7f5e6c0d06b0 con 0x7f5e6c160bb0 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.649+0000 7f5e61d9b640 1 -- 172.21.15.31:0/284508072 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0d2150 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.649+0000 7f5e61d9b640 1 --2- 172.21.15.31:0/284508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0d0170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009870 tx=0x7f5e5402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a765bc8d72e7a24d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e69d9b640 1 -- 172.21.15.31:0/284508072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54030860 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e69d9b640 1 -- 172.21.15.31:0/284508072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e540309c0 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e69d9b640 1 -- 172.21.15.31:0/284508072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54030d50 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e70914640 1 -- 172.21.15.31:0/284508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 msgr2=0x7f5e6c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e70914640 1 --2- 172.21.15.31:0/284508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0d0170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009870 tx=0x7f5e5402b9a0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e70914640 1 -- 172.21.15.31:0/284508072 shutdown_connections 2024-01-19T04:55:21.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e70914640 1 --2- 172.21.15.31:0/284508072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0d0170 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e70914640 1 -- 172.21.15.31:0/284508072 >> 172.21.15.31:0/284508072 conn(0x7f5e6c09c600 msgr2=0x7f5e6c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:21.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e70914640 1 -- 172.21.15.31:0/284508072 shutdown_connections 2024-01-19T04:55:21.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.650+0000 7f5e70914640 1 -- 172.21.15.31:0/284508072 wait complete. 2024-01-19T04:55:21.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 Processor -- start 2024-01-19T04:55:21.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 -- start start 2024-01-19T04:55:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e6c0d0ef0 0x7f5e6c1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e6c160bb0 0x7f5e6c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c09db10 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c1f3570 con 0x7f5e6c160bb0 2024-01-19T04:55:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.652+0000 7f5e70914640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c1f3db0 con 0x7f5e6c0d0ef0 2024-01-19T04:55:21.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0b7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35902/0 (socket says 172.21.15.31:35902) 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 -- 172.21.15.31:0/4079010051 learned_addr learned my addr 172.21.15.31:0/4079010051 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e61d9b640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e6c0d0ef0 0x7f5e6c1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6ad9d640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e6c160bb0 0x7f5e6c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 -- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e6c0d0ef0 msgr2=0x7f5e6c1528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e6c0d0ef0 0x7f5e6c1528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 -- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e6c160bb0 msgr2=0x7f5e6c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e6c160bb0 0x7f5e6c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 -- 172.21.15.31:0/4079010051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540094d0 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6ad9d640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e6c160bb0 0x7f5e6c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e6a59c640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0b7900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c4d0 tx=0x7f5e5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e637fe640 1 -- 172.21.15.31:0/4079010051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0074e0 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e637fe640 1 -- 172.21.15.31:0/4079010051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c007660 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c0a07b0 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e637fe640 1 -- 172.21.15.31:0/4079010051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c004a20 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.653+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c1f3a40 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.654+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6c0696e0 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.655+0000 7f5e637fe640 1 -- 172.21.15.31:0/4079010051 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5e5c004b80 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.655+0000 7f5e637fe640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5e4c06aed0 0x7f5e4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.655+0000 7f5e637fe640 1 -- 172.21.15.31:0/4079010051 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e5c088b60 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.656+0000 7f5e61d9b640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5e4c06aed0 0x7f5e4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.656+0000 7f5e61d9b640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5e4c06aed0 0x7f5e4c06d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e54036000 tx=0x7f5e5403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:21.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.660+0000 7f5e637fe640 1 -- 172.21.15.31:0/4079010051 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5e5c053a30 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.877+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e6c08d3b0 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.877+0000 7f5e637fe640 1 -- 172.21.15.31:0/4079010051 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f5e5c018020 con 0x7f5e6c0cfd10 2024-01-19T04:55:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.879+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5e4c06aed0 msgr2=0x7f5e4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.879+0000 7f5e70914640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5e4c06aed0 0x7f5e4c06d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e54036000 tx=0x7f5e5403b040 comp rx=0 tx=0).stop 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.879+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 msgr2=0x7f5e6c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.879+0000 7f5e70914640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0b7900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c4d0 tx=0x7f5e5c00c990 comp rx=0 tx=0).stop 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 shutdown_connections 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5e4c06aed0 0x7f5e4c06d390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e6c160bb0 0x7f5e6c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e6c0d0ef0 0x7f5e6c1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 --2- 172.21.15.31:0/4079010051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6c0cfd10 0x7f5e6c0b7900 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 >> 172.21.15.31:0/4079010051 conn(0x7f5e6c09c600 msgr2=0x7f5e6c0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:21.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 shutdown_connections 2024-01-19T04:55:21.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.880+0000 7f5e70914640 1 -- 172.21.15.31:0/4079010051 wait complete. 2024-01-19T04:55:21.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:21.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.991+0000 7f3eb5d83640 1 Processor -- start 2024-01-19T04:55:21.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.991+0000 7f3eb5d83640 1 -- start start 2024-01-19T04:55:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eb5d83640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3eb00b5410 0x7f3eb00b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eb5d83640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb0160df0 con 0x7f3eb00d2e50 2024-01-19T04:55:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eb5d83640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00b7300 con 0x7f3eb0160a10 2024-01-19T04:55:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eb5d83640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb00d1db0 con 0x7f3eb00b5410 2024-01-19T04:55:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eaffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3eb00b5410 0x7f3eb00b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eaffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3eb00b5410 0x7f3eb00b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35914/0 (socket says 172.21.15.31:35914) 2024-01-19T04:55:21.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eaeffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3eb00d2e50 0x7f3eb00b4d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56812/0 (socket says 172.21.15.31:56812) 2024-01-19T04:55:21.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eaffff640 1 -- 172.21.15.31:0/2559553332 learned_addr learned my addr 172.21.15.31:0/2559553332 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:21.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eaffff640 1 -- 172.21.15.31:0/2559553332 >> v1:172.21.15.137:6789/0 conn(0x7f3eb0160a10 legacy=0x7f3eb00d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:21.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eaffff640 1 -- 172.21.15.31:0/2559553332 >> v1:172.21.15.31:6789/0 conn(0x7f3eb00d2e50 legacy=0x7f3eb00b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.992+0000 7f3eaffff640 1 -- 172.21.15.31:0/2559553332 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00d2090 con 0x7f3eb00b5410 2024-01-19T04:55:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.993+0000 7f3eaffff640 1 --2- 172.21.15.31:0/2559553332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00b5410 0x7f3eb00b8ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009e50 tx=0x7f3ea802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e418af4690dfaec server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.993+0000 7f3eae7fc640 1 -- 172.21.15.31:0/2559553332 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea802cb20 con 0x7f3eb00b5410 2024-01-19T04:55:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.993+0000 7f3eae7fc640 1 -- 172.21.15.31:0/2559553332 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea802cc80 con 0x7f3eb00b5410 2024-01-19T04:55:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.993+0000 7f3eae7fc640 1 -- 172.21.15.31:0/2559553332 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80343f0 con 0x7f3eb00b5410 2024-01-19T04:55:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.994+0000 7f3eb5d83640 1 -- 172.21.15.31:0/2559553332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00b5410 msgr2=0x7f3eb00b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.994+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/2559553332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00b5410 0x7f3eb00b8ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009e50 tx=0x7f3ea802bf80 comp rx=0 tx=0).stop 2024-01-19T04:55:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.994+0000 7f3eb5d83640 1 -- 172.21.15.31:0/2559553332 shutdown_connections 2024-01-19T04:55:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.994+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/2559553332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00b5410 0x7f3eb00b8ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.994+0000 7f3eb5d83640 1 -- 172.21.15.31:0/2559553332 >> 172.21.15.31:0/2559553332 conn(0x7f3eb0072310 msgr2=0x7f3eb00cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.994+0000 7f3eb5d83640 1 -- 172.21.15.31:0/2559553332 shutdown_connections 2024-01-19T04:55:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.994+0000 7f3eb5d83640 1 -- 172.21.15.31:0/2559553332 wait complete. 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.995+0000 7f3eb5d83640 1 Processor -- start 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.995+0000 7f3eb5d83640 1 -- start start 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eb5d83640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb00b5410 0x7f3eb0152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eb5d83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00d2e50 0x7f3eb0154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eb5d83640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3eb0160a10 0x7f3eb01547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eb5d83640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0152e70 con 0x7f3eb00d2e50 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eb5d83640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0156850 con 0x7f3eb00b5410 2024-01-19T04:55:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eb5d83640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c80d0 con 0x7f3eb0160a10 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00d2e50 0x7f3eb0154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00d2e50 0x7f3eb0154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35928/0 (socket says 172.21.15.31:35928) 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 -- 172.21.15.31:0/3666279058 learned_addr learned my addr 172.21.15.31:0/3666279058 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 -- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3eb0160a10 msgr2=0x7f3eb01547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3eb0160a10 0x7f3eb01547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 -- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb00b5410 msgr2=0x7f3eb0152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaf7fe640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb00b5410 0x7f3eb0152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb00b5410 0x7f3eb0152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 -- 172.21.15.31:0/3666279058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8009b30 con 0x7f3eb00d2e50 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.996+0000 7f3eaeffd640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00d2e50 0x7f3eb0154270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c3a0 tx=0x7f3ea400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.997+0000 7f3e97fff640 1 -- 172.21.15.31:0/3666279058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40074e0 con 0x7f3eb00d2e50 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.997+0000 7f3e97fff640 1 -- 172.21.15.31:0/3666279058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4007640 con 0x7f3eb00d2e50 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.997+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb00c83b0 con 0x7f3eb00d2e50 2024-01-19T04:55:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.997+0000 7f3e97fff640 1 -- 172.21.15.31:0/3666279058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4004a00 con 0x7f3eb00d2e50 2024-01-19T04:55:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.997+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb00ca350 con 0x7f3eb00d2e50 2024-01-19T04:55:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.998+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb00c88d0 con 0x7f3eb00d2e50 2024-01-19T04:55:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.998+0000 7f3e97fff640 1 -- 172.21.15.31:0/3666279058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3ea4004b60 con 0x7f3eb00d2e50 2024-01-19T04:55:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.998+0000 7f3e97fff640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e9c06aed0 0x7f3e9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.999+0000 7f3e97fff640 1 -- 172.21.15.31:0/3666279058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ea4088fe0 con 0x7f3eb00d2e50 2024-01-19T04:55:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.999+0000 7f3eaf7fe640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e9c06aed0 0x7f3e9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:21.999+0000 7f3eaf7fe640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e9c06aed0 0x7f3e9c06d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0006fd0 tx=0x7f3ea0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.003+0000 7f3e97fff640 1 -- 172.21.15.31:0/3666279058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3eb00c88d0 con 0x7f3eb00d2e50 2024-01-19T04:55:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.183+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3eb00c88d0 con 0x7f3eb00d2e50 2024-01-19T04:55:22.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.184+0000 7f3e97fff640 1 -- 172.21.15.31:0/3666279058 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f3eb00c88d0 con 0x7f3eb00d2e50 2024-01-19T04:55:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:22.233 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:22.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e9c06aed0 msgr2=0x7f3e9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e9c06aed0 0x7f3e9c06d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0006fd0 tx=0x7f3ea0008040 comp rx=0 tx=0).stop 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00d2e50 msgr2=0x7f3eb0154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00d2e50 0x7f3eb0154270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c3a0 tx=0x7f3ea400c860 comp rx=0 tx=0).stop 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 shutdown_connections 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e9c06aed0 0x7f3e9c06d390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3eb0160a10 0x7f3eb01547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00d2e50 0x7f3eb0154270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 --2- 172.21.15.31:0/3666279058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb00b5410 0x7f3eb0152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 >> 172.21.15.31:0/3666279058 conn(0x7f3eb0072310 msgr2=0x7f3eb00cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 shutdown_connections 2024-01-19T04:55:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.186+0000 7f3eb5d83640 1 -- 172.21.15.31:0/3666279058 wait complete. 2024-01-19T04:55:22.236 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-19T04:55:22.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:55:22.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.337+0000 7f2c8df10640 1 Processor -- start 2024-01-19T04:55:22.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.337+0000 7f2c8df10640 1 -- start start 2024-01-19T04:55:22.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.338+0000 7f2c8df10640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2c8815d540 0x7f2c880d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.338+0000 7f2c8df10640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8815d920 con 0x7f2c880cda90 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.338+0000 7f2c8df10640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8806c040 con 0x7f2c8806baa0 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.338+0000 7f2c8df10640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8806c320 con 0x7f2c8815d540 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.338+0000 7f2c87fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c880cda90 0x7f2c880cdea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56826/0 (socket says 172.21.15.31:56826) 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.338+0000 7f2c87fff640 1 -- 172.21.15.31:0/1298565732 learned_addr learned my addr 172.21.15.31:0/1298565732 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 --2- 172.21.15.31:0/1298565732 >> v2:172.21.15.137:3300/0 conn(0x7f2c8815d540 0x7f2c880d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419258392 0 0) 0x7f2c8815d920 con 0x7f2c880cda90 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c70003520 con 0x7f2c880cda90 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4006348841 0 0) 0x7f2c70003520 con 0x7f2c880cda90 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 >> v2:172.21.15.137:3300/0 conn(0x7f2c8815d540 msgr2=0x7f2c880d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 --2- 172.21.15.31:0/1298565732 >> v2:172.21.15.137:3300/0 conn(0x7f2c8815d540 0x7f2c880d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 >> v1:172.21.15.137:6789/0 conn(0x7f2c8806baa0 legacy=0x7f2c880cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8806c600 con 0x7f2c880cda90 2024-01-19T04:55:22.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 --2- 172.21.15.31:0/1298565732 >> v2:172.21.15.137:3300/0 conn(0x7f2c8815d540 0x7f2c880d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f2c800019f0 con 0x7f2c880cda90 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 >> v1:172.21.15.31:6789/0 conn(0x7f2c880cda90 legacy=0x7f2c880cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c70003970 0x7f2c70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c700047c0 0x7f2c70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70005180 0x7f2c7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8806c320 con 0x7f2c70003970 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7000c020 con 0x7f2c700047c0 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7000c1e0 con 0x7f2c70005180 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c70003970 0x7f2c70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c86ffd640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c700047c0 0x7f2c70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c87fff640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70005180 0x7f2c7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 -- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70005180 msgr2=0x7f2c7000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70005180 0x7f2c7000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 -- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c700047c0 msgr2=0x7f2c70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c700047c0 0x7f2c70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c877fe640 1 -- 172.21.15.31:0/1298565732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c70003520 con 0x7f2c70003970 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c87fff640 1 -- 172.21.15.31:0/1298565732 reap_dead start 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.339+0000 7f2c86ffd640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c700047c0 0x7f2c70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.340+0000 7f2c877fe640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c70003970 0x7f2c70003d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009c90 tx=0x7f2c700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba70958f812a356 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.340+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7802c910 con 0x7f2c70003970 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.340+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7802ca70 con 0x7f2c70003970 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.340+0000 7f2c867fc640 1 -- 172.21.15.31:0/1298565732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7802ce60 con 0x7f2c70003970 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.341+0000 7f2c8df10640 1 -- 172.21.15.31:0/1298565732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c70003970 msgr2=0x7f2c70003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.341+0000 7f2c8df10640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c70003970 0x7f2c70003d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009c90 tx=0x7f2c700057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.341+0000 7f2c8df10640 1 -- 172.21.15.31:0/1298565732 shutdown_connections 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.341+0000 7f2c8df10640 1 --2- 172.21.15.31:0/1298565732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c70003970 0x7f2c70003d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.341+0000 7f2c8df10640 1 -- 172.21.15.31:0/1298565732 >> 172.21.15.31:0/1298565732 conn(0x7f2c880b2c60 msgr2=0x7f2c880b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.341+0000 7f2c8df10640 1 -- 172.21.15.31:0/1298565732 shutdown_connections 2024-01-19T04:55:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.341+0000 7f2c8df10640 1 -- 172.21.15.31:0/1298565732 wait complete. 2024-01-19T04:55:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.342+0000 7f2c8df10640 1 Processor -- start 2024-01-19T04:55:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.342+0000 7f2c8df10640 1 -- start start 2024-01-19T04:55:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c8df10640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70003970 0x7f2c880c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c8df10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c880c4b90 0x7f2c880c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c8df10640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8814b700 0x7f2c8814bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c8df10640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8814c0c0 con 0x7f2c880c4b90 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c8df10640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8814c350 con 0x7f2c8814b700 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c8df10640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880c9620 con 0x7f2c70003970 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c880c4b90 0x7f2c880c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c880c4b90 0x7f2c880c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35956/0 (socket says 172.21.15.31:35956) 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 -- 172.21.15.31:0/64536003 learned_addr learned my addr 172.21.15.31:0/64536003 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c87fff640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8814b700 0x7f2c8814bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c877fe640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70003970 0x7f2c880c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 -- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70003970 msgr2=0x7f2c880c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70003970 0x7f2c880c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 -- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8814b700 msgr2=0x7f2c8814bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8814b700 0x7f2c8814bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 -- 172.21.15.31:0/64536003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880c9900 con 0x7f2c880c4b90 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.343+0000 7f2c86ffd640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c880c4b90 0x7f2c880c7610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e480 tx=0x7f2c7c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.344+0000 7f2c6ffff640 1 -- 172.21.15.31:0/64536003 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c017070 con 0x7f2c880c4b90 2024-01-19T04:55:22.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.344+0000 7f2c6ffff640 1 -- 172.21.15.31:0/64536003 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c0052e0 con 0x7f2c880c4b90 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.344+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c880c9be0 con 0x7f2c880c4b90 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.344+0000 7f2c6ffff640 1 -- 172.21.15.31:0/64536003 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c005490 con 0x7f2c880c4b90 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.344+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c880c5010 con 0x7f2c880c4b90 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.345+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c88074a80 con 0x7f2c880c4b90 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.345+0000 7f2c6ffff640 1 -- 172.21.15.31:0/64536003 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2c7c007020 con 0x7f2c880c4b90 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.345+0000 7f2c6ffff640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2c7406aed0 0x7f2c7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.345+0000 7f2c6ffff640 1 -- 172.21.15.31:0/64536003 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c7c088a60 con 0x7f2c880c4b90 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.346+0000 7f2c877fe640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2c7406aed0 0x7f2c7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.346+0000 7f2c877fe640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2c7406aed0 0x7f2c7406d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c78036000 tx=0x7f2c78030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:22.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.350+0000 7f2c6ffff640 1 -- 172.21.15.31:0/64536003 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2c7c053a70 con 0x7f2c880c4b90 2024-01-19T04:55:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.528+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c8806d460 con 0x7f2c7406aed0 2024-01-19T04:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.536+0000 7f2c6ffff640 1 -- 172.21.15.31:0/64536003 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249793 (secure 0 0 0) 0x7f2c78006e30 con 0x7f2c7406aed0 2024-01-19T04:55:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2c7406aed0 msgr2=0x7f2c7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2c7406aed0 0x7f2c7406d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c78036000 tx=0x7f2c78030000 comp rx=0 tx=0).stop 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c880c4b90 msgr2=0x7f2c880c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c880c4b90 0x7f2c880c7610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e480 tx=0x7f2c7c00e940 comp rx=0 tx=0).stop 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 shutdown_connections 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2c7406aed0 0x7f2c7406d390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8814b700 0x7f2c8814bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c880c4b90 0x7f2c880c7610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 --2- 172.21.15.31:0/64536003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2c70003970 0x7f2c880c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 >> 172.21.15.31:0/64536003 conn(0x7f2c880b2c60 msgr2=0x7f2c880b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 shutdown_connections 2024-01-19T04:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.539+0000 7f2c8df10640 1 -- 172.21.15.31:0/64536003 wait complete. 2024-01-19T04:55:22.549 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-19T04:55:21.561553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218824,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":2433024,"data_stored":1546680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.058484"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-01-19T04:55:09.209541+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:55:09.209541+0000","last_peered":"2024-01-19T04:55:09.209541+0000","last_clean":"2024-01-19T04:55:09.209541+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:55:09.209541+0000","last_undegraded":"2024-01-19T04:55:09.209541+0000","last_fullsized":"2024-01-19T04:55:09.209541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:55:22.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:55:22.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.691+0000 7f0c30215640 1 Processor -- start 2024-01-19T04:55:22.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.691+0000 7f0c30215640 1 -- start start 2024-01-19T04:55:22.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c30215640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 0x7f0c280b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c30215640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c28160df0 con 0x7f0c280b4d00 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c30215640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c280d1db0 con 0x7f0c280d2e50 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c30215640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280d2090 con 0x7f0c28160a10 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c2e7b2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0c280b4d00 0x7f0c280b62f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56842/0 (socket says 172.21.15.31:56842) 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c2e7b2640 1 -- 172.21.15.31:0/631468574 learned_addr learned my addr 172.21.15.31:0/631468574 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c2dfb1640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 0x7f0c280b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779378136 0 0) 0x7f0c28160df0 con 0x7f0c280b4d00 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.692+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c10003520 con 0x7f0c280b4d00 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2866983021 0 0) 0x7f0c10003520 con 0x7f0c280b4d00 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 msgr2=0x7f0c280b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 0x7f0c280b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 >> v1:172.21.15.137:6789/0 conn(0x7f0c280d2e50 legacy=0x7f0c280d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280d2370 con 0x7f0c280b4d00 2024-01-19T04:55:22.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f0c180019f0 con 0x7f0c280b4d00 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 >> v1:172.21.15.31:6789/0 conn(0x7f0c280b4d00 legacy=0x7f0c280b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c100039a0 0x7f0c10003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c10004350 0x7f0c10004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 0x7f0c10005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280d2090 con 0x7f0c10004350 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1000b980 con 0x7f0c100039a0 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1000bb40 con 0x7f0c28160a10 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2d7b0640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c10004350 0x7f0c10004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2dfb1640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c100039a0 0x7f0c10003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2e7b2640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 0x7f0c10005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2d7b0640 1 -- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 msgr2=0x7f0c10005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2d7b0640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28160a10 0x7f0c10005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2d7b0640 1 -- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c100039a0 msgr2=0x7f0c10003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:22.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2d7b0640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c100039a0 0x7f0c10003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.693+0000 7f0c2d7b0640 1 -- 172.21.15.31:0/631468574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10003520 con 0x7f0c10004350 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c2d7b0640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c10004350 0x7f0c10004820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c001830 tx=0x7f0c100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39171cde5474f1ad server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c02f050 con 0x7f0c10004350 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c033610 con 0x7f0c10004350 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c2cfaf640 1 -- 172.21.15.31:0/631468574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0339c0 con 0x7f0c10004350 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c30215640 1 -- 172.21.15.31:0/631468574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c10004350 msgr2=0x7f0c10004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c30215640 1 --2- 172.21.15.31:0/631468574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c10004350 0x7f0c10004820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c001830 tx=0x7f0c100057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c2e7b2640 1 -- 172.21.15.31:0/631468574 reap_dead start 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c30215640 1 -- 172.21.15.31:0/631468574 shutdown_connections 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.694+0000 7f0c30215640 1 -- 172.21.15.31:0/631468574 >> 172.21.15.31:0/631468574 conn(0x7f0c28072310 msgr2=0x7f0c280cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.695+0000 7f0c30215640 1 -- 172.21.15.31:0/631468574 shutdown_connections 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.695+0000 7f0c30215640 1 -- 172.21.15.31:0/631468574 wait complete. 2024-01-19T04:55:22.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 Processor -- start 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 -- start start 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10004350 0x7f0c280ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c281f6010 0x7f0c2814dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c280b00f0 0x7f0c280cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28154ac0 con 0x7f0c280b00f0 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281f28f0 con 0x7f0c10004350 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.696+0000 7f0c30215640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280abc20 con 0x7f0c281f6010 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c280b00f0 0x7f0c280cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c280b00f0 0x7f0c280cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35980/0 (socket says 172.21.15.31:35980) 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 -- 172.21.15.31:0/762162515 learned_addr learned my addr 172.21.15.31:0/762162515 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2dfb1640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10004350 0x7f0c280ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2d7b0640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c281f6010 0x7f0c2814dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 -- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c281f6010 msgr2=0x7f0c2814dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c281f6010 0x7f0c2814dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 -- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10004350 msgr2=0x7f0c280ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10004350 0x7f0c280ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 -- 172.21.15.31:0/762162515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c281f6580 con 0x7f0c280b00f0 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2dfb1640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10004350 0x7f0c280ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c2e7b2640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c280b00f0 0x7f0c280cc520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000f80 tx=0x7f0c1800a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c267fc640 1 -- 172.21.15.31:0/762162515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800acf0 con 0x7f0c280b00f0 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c267fc640 1 -- 172.21.15.31:0/762162515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1800ae70 con 0x7f0c280b00f0 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c281f6860 con 0x7f0c280b00f0 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c267fc640 1 -- 172.21.15.31:0/762162515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18013610 con 0x7f0c280b00f0 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.697+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c281f6e20 con 0x7f0c280b00f0 2024-01-19T04:55:22.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.698+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c280696e0 con 0x7f0c280b00f0 2024-01-19T04:55:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.699+0000 7f0c267fc640 1 -- 172.21.15.31:0/762162515 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0c18013770 con 0x7f0c280b00f0 2024-01-19T04:55:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.699+0000 7f0c267fc640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406af00 0x7f0c1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.699+0000 7f0c267fc640 1 -- 172.21.15.31:0/762162515 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c1808a000 con 0x7f0c280b00f0 2024-01-19T04:55:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.699+0000 7f0c2dfb1640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406af00 0x7f0c1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.700+0000 7f0c2dfb1640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406af00 0x7f0c1406d3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009a40 tx=0x7f0c20008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:22.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.703+0000 7f0c267fc640 1 -- 172.21.15.31:0/762162515 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0c180589f0 con 0x7f0c280b00f0 2024-01-19T04:55:22.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.879+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c280d2930 con 0x7f0c1406af00 2024-01-19T04:55:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.886+0000 7f0c267fc640 1 -- 172.21.15.31:0/762162515 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249793 (secure 0 0 0) 0x7f0c20013a20 con 0x7f0c1406af00 2024-01-19T04:55:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:22.889 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:55:22.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406af00 msgr2=0x7f0c1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406af00 0x7f0c1406d3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009a40 tx=0x7f0c20008040 comp rx=0 tx=0).stop 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c280b00f0 msgr2=0x7f0c280cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c280b00f0 0x7f0c280cc520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000f80 tx=0x7f0c1800a6f0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 shutdown_connections 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0c1406af00 0x7f0c1406d3c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c280b00f0 0x7f0c280cc520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c281f6010 0x7f0c2814dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 --2- 172.21.15.31:0/762162515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10004350 0x7f0c280ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 >> 172.21.15.31:0/762162515 conn(0x7f0c28072310 msgr2=0x7f0c2806b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 shutdown_connections 2024-01-19T04:55:22.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:22.889+0000 7f0c30215640 1 -- 172.21.15.31:0/762162515 wait complete. 2024-01-19T04:55:22.899 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-19T04:55:21.561553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218824,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":2433024,"data_stored":1546680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.058484"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-01-19T04:55:09.209541+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:55:09.209541+0000","last_peered":"2024-01-19T04:55:09.209541+0000","last_clean":"2024-01-19T04:55:09.209541+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:55:09.209541+0000","last_undegraded":"2024-01-19T04:55:09.209541+0000","last_fullsized":"2024-01-19T04:55:09.209541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:55:22.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:55:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.040+0000 7f3be2ba4640 1 Processor -- start 2024-01-19T04:55:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be2ba4640 1 -- start start 2024-01-19T04:55:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be2ba4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bdc15d8a0 0x7f3bdc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be2ba4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3bdc15f0c0 0x7f3bdc15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be2ba4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc06bf30 con 0x7f3bdc15eb70 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be2ba4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc15fa30 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be2ba4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc0cb9e0 con 0x7f3bdc15f0c0 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be0940640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bdc15d8a0 0x7f3bdc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.041+0000 7f3be0940640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bdc15d8a0 0x7f3bdc06b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35990/0 (socket says 172.21.15.31:35990) 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.042+0000 7f3be0940640 1 -- 172.21.15.31:0/2960220983 learned_addr learned my addr 172.21.15.31:0/2960220983 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.042+0000 7f3be1141640 1 --2- 172.21.15.31:0/2960220983 >> v2:172.21.15.137:3300/0 conn(0x7f3bdc15f0c0 0x7f3bdc15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.042+0000 7f3be0940640 1 -- 172.21.15.31:0/2960220983 >> v2:172.21.15.137:3300/0 conn(0x7f3bdc15f0c0 msgr2=0x7f3bdc15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.042+0000 7f3be0940640 1 --2- 172.21.15.31:0/2960220983 >> v2:172.21.15.137:3300/0 conn(0x7f3bdc15f0c0 0x7f3bdc15f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.042+0000 7f3be0940640 1 -- 172.21.15.31:0/2960220983 >> v1:172.21.15.137:6789/0 conn(0x7f3bdc15eb70 legacy=0x7f3bdc0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.042+0000 7f3be0940640 1 -- 172.21.15.31:0/2960220983 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0cbcc0 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.042+0000 7f3be1141640 1 --2- 172.21.15.31:0/2960220983 >> v2:172.21.15.137:3300/0 conn(0x7f3bdc15f0c0 0x7f3bdc15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be0940640 1 --2- 172.21.15.31:0/2960220983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc06b6c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc06d260 tx=0x7f3bd802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da65a0c122a6ecf2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3bd37fe640 1 -- 172.21.15.31:0/2960220983 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8031870 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3bd37fe640 1 -- 172.21.15.31:0/2960220983 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd80319d0 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3bd37fe640 1 -- 172.21.15.31:0/2960220983 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8031d60 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 -- 172.21.15.31:0/2960220983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 msgr2=0x7f3bdc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/2960220983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc06b6c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc06d260 tx=0x7f3bd802b910 comp rx=0 tx=0).stop 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 -- 172.21.15.31:0/2960220983 shutdown_connections 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/2960220983 >> v2:172.21.15.137:3300/0 conn(0x7f3bdc15f0c0 0x7f3bdc15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/2960220983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc06b6c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 -- 172.21.15.31:0/2960220983 >> 172.21.15.31:0/2960220983 conn(0x7f3bdc06abe0 msgr2=0x7f3bdc066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 -- 172.21.15.31:0/2960220983 shutdown_connections 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.043+0000 7f3be2ba4640 1 -- 172.21.15.31:0/2960220983 wait complete. 2024-01-19T04:55:23.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.044+0000 7f3be2ba4640 1 Processor -- start 2024-01-19T04:55:23.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.044+0000 7f3be2ba4640 1 -- start start 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be2ba4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc0b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be2ba4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bdc15eb70 0x7f3bdc0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be2ba4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bdc15f0c0 0x7f3bdc0d1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be2ba4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0b6e50 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be2ba4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0cb6b0 con 0x7f3bdc15eb70 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be2ba4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0ca3b0 con 0x7f3bdc15f0c0 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be0940640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc0b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be0940640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc0b6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36002/0 (socket says 172.21.15.31:36002) 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.045+0000 7f3be0940640 1 -- 172.21.15.31:0/538883945 learned_addr learned my addr 172.21.15.31:0/538883945 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be1141640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bdc15f0c0 0x7f3bdc0d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3bd3fff640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bdc15eb70 0x7f3bdc0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be0940640 1 -- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bdc15f0c0 msgr2=0x7f3bdc0d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be0940640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bdc15f0c0 0x7f3bdc0d1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be0940640 1 -- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bdc15eb70 msgr2=0x7f3bdc0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be0940640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bdc15eb70 0x7f3bdc0cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be0940640 1 -- 172.21.15.31:0/538883945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80094d0 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be0940640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc0b6910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3bd802b950 tx=0x7f3bd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3bd17fa640 1 -- 172.21.15.31:0/538883945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd803ad30 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3bd17fa640 1 -- 172.21.15.31:0/538883945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd80443f0 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc0ca690 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3bd17fa640 1 -- 172.21.15.31:0/538883945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80447a0 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.046+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc0cac50 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.047+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc0696e0 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.047+0000 7f3bd17fa640 1 -- 172.21.15.31:0/538883945 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3bd804d450 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.048+0000 7f3bd17fa640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3bc806af00 0x7f3bc806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.048+0000 7f3bd17fa640 1 -- 172.21.15.31:0/538883945 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bd802d040 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.048+0000 7f3bd3fff640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3bc806af00 0x7f3bc806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.048+0000 7f3bd3fff640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3bc806af00 0x7f3bc806d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc007fd0 tx=0x7f3bcc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.052+0000 7f3bd17fa640 1 -- 172.21.15.31:0/538883945 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3bd8076a90 con 0x7f3bdc15d8a0 2024-01-19T04:55:23.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.227+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bdc06bc00 con 0x7f3bc806af00 2024-01-19T04:55:23.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.233+0000 7f3bd17fa640 1 -- 172.21.15.31:0/538883945 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249793 (secure 0 0 0) 0x7f3bcc0050d0 con 0x7f3bc806af00 2024-01-19T04:55:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:23.237 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:55:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3bc806af00 msgr2=0x7f3bc806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3bc806af00 0x7f3bc806d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc007fd0 tx=0x7f3bcc009040 comp rx=0 tx=0).stop 2024-01-19T04:55:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 msgr2=0x7f3bdc0b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc0b6910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3bd802b950 tx=0x7f3bd80046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 shutdown_connections 2024-01-19T04:55:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3bc806af00 0x7f3bc806d3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bdc15f0c0 0x7f3bdc0d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bdc15eb70 0x7f3bdc0cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 --2- 172.21.15.31:0/538883945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bdc15d8a0 0x7f3bdc0b6910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 >> 172.21.15.31:0/538883945 conn(0x7f3bdc06abe0 msgr2=0x7f3bdc0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 shutdown_connections 2024-01-19T04:55:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.236+0000 7f3be2ba4640 1 -- 172.21.15.31:0/538883945 wait complete. 2024-01-19T04:55:23.246 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-19T04:55:21.561553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218824,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":2433024,"data_stored":1546680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.058484"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-01-19T04:55:09.209541+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:55:09.209541+0000","last_peered":"2024-01-19T04:55:09.209541+0000","last_clean":"2024-01-19T04:55:09.209541+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:55:09.209541+0000","last_undegraded":"2024-01-19T04:55:09.209541+0000","last_fullsized":"2024-01-19T04:55:09.209541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:18.503250+0000","last_change":"2024-01-19T04:55:18.503250+0000","last_active":"2024-01-19T04:55:18.503250+0000","last_peered":"2024-01-19T04:55:18.503250+0000","last_clean":"2024-01-19T04:55:18.503250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:18.503250+0000","last_undegraded":"2024-01-19T04:55:18.503250+0000","last_fullsized":"2024-01-19T04:55:18.503250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:55:17.497959+0000","last_change":"2024-01-19T04:55:17.497959+0000","last_active":"2024-01-19T04:55:17.497959+0000","last_peered":"2024-01-19T04:55:17.497959+0000","last_clean":"2024-01-19T04:55:17.497959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:55:17.497959+0000","last_undegraded":"2024-01-19T04:55:17.497959+0000","last_fullsized":"2024-01-19T04:55:17.497959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:55:23.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-01-19T04:55:23.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-01-19T04:55:23.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-01-19T04:55:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-01-19T04:55:23.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-01-19T04:55:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-01-19T04:55:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-01-19T04:55:23.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:18.503250+0000', 'last_change': '2024-01-19T04:55:18.503250+0000', 'last_active': '2024-01-19T04:55:18.503250+0000', 'last_peered': '2024-01-19T04:55:18.503250+0000', 'last_clean': '2024-01-19T04:55:18.503250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:18.503250+0000', 'last_undegraded': '2024-01-19T04:55:18.503250+0000', 'last_fullsized': '2024-01-19T04:55:18.503250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:18.503250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-01-19T04:55:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-19T04:55:17.497959+0000', 'last_change': '2024-01-19T04:55:17.497959+0000', 'last_active': '2024-01-19T04:55:17.497959+0000', 'last_peered': '2024-01-19T04:55:17.497959+0000', 'last_clean': '2024-01-19T04:55:17.497959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-19T04:55:17.497959+0000', 'last_undegraded': '2024-01-19T04:55:17.497959+0000', 'last_fullsized': '2024-01-19T04:55:17.497959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'last_clean_scrub_stamp': '2024-01-19T04:55:17.497959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-19T04:55:23.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-19T04:55:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.386+0000 7f3754543640 1 Processor -- start 2024-01-19T04:55:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.387+0000 7f3754543640 1 -- start start 2024-01-19T04:55:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.387+0000 7f3754543640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.387+0000 7f3754543640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f374c0cfd10 0x7f374c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.387+0000 7f3754543640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374c0d06b0 con 0x7f374c0d0ef0 2024-01-19T04:55:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.387+0000 7f3754543640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374c0b66b0 con 0x7f374c0cfd10 2024-01-19T04:55:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.387+0000 7f3754543640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0d1b90 con 0x7f374c160bb0 2024-01-19T04:55:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3751ade640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f374c0cfd10 0x7f374c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f37522df640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3751ade640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f374c0cfd10 0x7f374c0d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34682/0 (socket says 172.21.15.31:34682) 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3751ade640 1 -- 172.21.15.31:0/1468752847 learned_addr learned my addr 172.21.15.31:0/1468752847 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3752ae0640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f374c0d0ef0 0x7f374c0d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:46324/0 (socket says 172.21.15.31:46324) 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3751ade640 1 -- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 msgr2=0x7f374c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3751ade640 1 --2- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c0b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3751ade640 1 -- 172.21.15.31:0/1468752847 >> v1:172.21.15.137:6789/0 conn(0x7f374c0d0ef0 legacy=0x7f374c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f3751ade640 1 -- 172.21.15.31:0/1468752847 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0d1e70 con 0x7f374c0cfd10 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.388+0000 7f37522df640 1 --2- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c0b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.389+0000 7f3751ade640 1 --2- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 0x7f374c0d0170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c4ccf6cdc866e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.389+0000 7f37512dd640 1 -- 172.21.15.31:0/1468752847 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740030840 con 0x7f374c0cfd10 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.389+0000 7f37512dd640 1 -- 172.21.15.31:0/1468752847 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37400309a0 con 0x7f374c0cfd10 2024-01-19T04:55:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.389+0000 7f37512dd640 1 -- 172.21.15.31:0/1468752847 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740030d30 con 0x7f374c0cfd10 2024-01-19T04:55:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 -- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 msgr2=0x7f374c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 --2- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 0x7f374c0d0170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374002b980 comp rx=0 tx=0).stop 2024-01-19T04:55:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 -- 172.21.15.31:0/1468752847 shutdown_connections 2024-01-19T04:55:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 --2- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 0x7f374c0d0170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 --2- 172.21.15.31:0/1468752847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c0b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 -- 172.21.15.31:0/1468752847 >> 172.21.15.31:0/1468752847 conn(0x7f374c09c600 msgr2=0x7f374c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 -- 172.21.15.31:0/1468752847 shutdown_connections 2024-01-19T04:55:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.390+0000 7f3754543640 1 -- 172.21.15.31:0/1468752847 wait complete. 2024-01-19T04:55:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.391+0000 7f3754543640 1 Processor -- start 2024-01-19T04:55:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.391+0000 7f3754543640 1 -- start start 2024-01-19T04:55:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 0x7f374c0c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c0d0ef0 0x7f374c0c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0ca230 con 0x7f374c0d0ef0 2024-01-19T04:55:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c149060 con 0x7f374c0cfd10 2024-01-19T04:55:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0c91b0 con 0x7f374c160bb0 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c0d0ef0 0x7f374c0c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c0d0ef0 0x7f374c0c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36014/0 (socket says 172.21.15.31:36014) 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 -- 172.21.15.31:0/2919371213 learned_addr learned my addr 172.21.15.31:0/2919371213 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f37522df640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 0x7f374c0c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3752ae0640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 -- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 msgr2=0x7f374c14b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c14b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 -- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 msgr2=0x7f374c0c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 0x7f374c0c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 -- 172.21.15.31:0/2919371213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744009400 con 0x7f374c0d0ef0 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3751ade640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c0d0ef0 0x7f374c0c7390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3740000c00 tx=0x7f3740004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f374affd640 1 -- 172.21.15.31:0/2919371213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740039a90 con 0x7f374c0d0ef0 2024-01-19T04:55:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f374affd640 1 -- 172.21.15.31:0/2919371213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3740039bf0 con 0x7f374c0d0ef0 2024-01-19T04:55:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37400094d0 con 0x7f374c0d0ef0 2024-01-19T04:55:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f374affd640 1 -- 172.21.15.31:0/2919371213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740043410 con 0x7f374c0d0ef0 2024-01-19T04:55:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.392+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c0c9a50 con 0x7f374c0d0ef0 2024-01-19T04:55:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.393+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374c0c95b0 con 0x7f374c0d0ef0 2024-01-19T04:55:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.394+0000 7f374affd640 1 -- 172.21.15.31:0/2919371213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f374004c3b0 con 0x7f374c0d0ef0 2024-01-19T04:55:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.394+0000 7f374affd640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f373806aed0 0x7f373806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.394+0000 7f374affd640 1 -- 172.21.15.31:0/2919371213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37400ab8e0 con 0x7f374c0d0ef0 2024-01-19T04:55:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.394+0000 7f37522df640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f373806aed0 0x7f373806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.395+0000 7f37522df640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f373806aed0 0x7f373806d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3744009cd0 tx=0x7f3744009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.398+0000 7f374affd640 1 -- 172.21.15.31:0/2919371213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f37400768f0 con 0x7f374c0d0ef0 2024-01-19T04:55:23.495 INFO:tasks.ceph.mds.a.smithi031.stderr:2024-01-19T04:55:23.495+0000 7f14d06fd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-19T04:55:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.644+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f374c14c310 con 0x7f374c0d0ef0 2024-01-19T04:55:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.645+0000 7f374affd640 1 -- 172.21.15.31:0/2919371213 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1021 (secure 0 0 0) 0x7f3740079f10 con 0x7f374c0d0ef0 2024-01-19T04:55:23.645 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:23.645 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1705640105,"num_in_osds":8,"osd_in_since":1705640096,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":224075776,"bytes_avail":772870037504,"bytes_total":773094113280,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4290}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-01-19T04:54:54.738185+0000","services":{}},"progress_events":{}} 2024-01-19T04:55:23.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f373806aed0 msgr2=0x7f373806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f373806aed0 0x7f373806d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3744009cd0 tx=0x7f3744009190 comp rx=0 tx=0).stop 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c0d0ef0 msgr2=0x7f374c0c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c0d0ef0 0x7f374c0c7390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3740000c00 tx=0x7f3740004730 comp rx=0 tx=0).stop 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 shutdown_connections 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f373806aed0 0x7f373806d390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374c160bb0 0x7f374c14b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c0d0ef0 0x7f374c0c7390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 --2- 172.21.15.31:0/2919371213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f374c0cfd10 0x7f374c0c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 >> 172.21.15.31:0/2919371213 conn(0x7f374c09c600 msgr2=0x7f374c0b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 shutdown_connections 2024-01-19T04:55:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.647+0000 7f3754543640 1 -- 172.21.15.31:0/2919371213 wait complete. 2024-01-19T04:55:23.657 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-19T04:55:23.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:55:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.758+0000 7f9edfa15640 1 Processor -- start 2024-01-19T04:55:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.758+0000 7f9edfa15640 1 -- start start 2024-01-19T04:55:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9edfa15640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9ed8160bb0 0x7f9ed80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9edfa15640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed80b66b0 con 0x7f9ed80cfd10 2024-01-19T04:55:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9edfa15640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed80d1b90 con 0x7f9ed80d1600 2024-01-19T04:55:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9edfa15640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed80d1e70 con 0x7f9ed8160bb0 2024-01-19T04:55:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9edcfb0640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9ed80cfd10 0x7f9ed80d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56848/0 (socket says 172.21.15.31:56848) 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9edcfb0640 1 -- 172.21.15.31:0/2210798284 learned_addr learned my addr 172.21.15.31:0/2210798284 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2210798284 >> v2:172.21.15.137:3300/0 conn(0x7f9ed8160bb0 0x7f9ed80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.759+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293128314 0 0) 0x7f9ed80b66b0 con 0x7f9ed80cfd10 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec0003520 con 0x7f9ed80cfd10 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 814564991 0 0) 0x7f9ec0003520 con 0x7f9ed80cfd10 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 >> v2:172.21.15.137:3300/0 conn(0x7f9ed8160bb0 msgr2=0x7f9ed80b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 --2- 172.21.15.31:0/2210798284 >> v2:172.21.15.137:3300/0 conn(0x7f9ed8160bb0 0x7f9ed80b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 >> v1:172.21.15.137:6789/0 conn(0x7f9ed80d1600 legacy=0x7f9ed80ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed80d2150 con 0x7f9ed80cfd10 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f9ecc001670 con 0x7f9ed80cfd10 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 >> v1:172.21.15.31:6789/0 conn(0x7f9ed80cfd10 legacy=0x7f9ed80d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ec0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ec00047c0 0x7f9ec0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec0005180 0x7f9ec000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80d1e70 con 0x7f9ec0003970 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec000c020 con 0x7f9ec0005180 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec000c1e0 con 0x7f9ec00047c0 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ec0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9edcfb0640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ec00047c0 0x7f9ec0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9eddfb2640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec0005180 0x7f9ec000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9edd7b1640 1 -- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ec00047c0 msgr2=0x7f9ec0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ec00047c0 0x7f9ec0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9edd7b1640 1 -- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec0005180 msgr2=0x7f9ec000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec0005180 0x7f9ec000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9edd7b1640 1 -- 172.21.15.31:0/2210798284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec0003520 con 0x7f9ec0003970 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.760+0000 7f9eddfb2640 1 -- 172.21.15.31:0/2210798284 reap_dead start 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.761+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ec0003d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009ac0 tx=0x7f9ec00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ee951a7947238 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.761+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed002c6b0 con 0x7f9ec0003970 2024-01-19T04:55:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.761+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed002c810 con 0x7f9ec0003970 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.761+0000 7f9ed7fff640 1 -- 172.21.15.31:0/2210798284 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed002cba0 con 0x7f9ec0003970 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.761+0000 7f9edfa15640 1 -- 172.21.15.31:0/2210798284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 msgr2=0x7f9ec0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.761+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ec0003d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009ac0 tx=0x7f9ec00057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.762+0000 7f9edfa15640 1 -- 172.21.15.31:0/2210798284 shutdown_connections 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.762+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2210798284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ec0003d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.762+0000 7f9edfa15640 1 -- 172.21.15.31:0/2210798284 >> 172.21.15.31:0/2210798284 conn(0x7f9ed809c600 msgr2=0x7f9ed80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.762+0000 7f9edfa15640 1 -- 172.21.15.31:0/2210798284 shutdown_connections 2024-01-19T04:55:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.762+0000 7f9edfa15640 1 -- 172.21.15.31:0/2210798284 wait complete. 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.763+0000 7f9edfa15640 1 Processor -- start 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.763+0000 7f9edfa15640 1 -- start start 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.763+0000 7f9edfa15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ed80b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.763+0000 7f9edfa15640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ed81f3180 0x7f9ed8157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edfa15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ed80cec60 0x7f9ed80cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edfa15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80cf650 con 0x7f9ec0003970 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edfa15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81f45e0 con 0x7f9ed80cec60 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edfa15640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81f39b0 con 0x7f9ed81f3180 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ed80b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ed80b8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36028/0 (socket says 172.21.15.31:36028) 2024-01-19T04:55:23.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 -- 172.21.15.31:0/2911916414 learned_addr learned my addr 172.21.15.31:0/2911916414 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9eddfb2640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ed80cec60 0x7f9ed80cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edcfb0640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ed81f3180 0x7f9ed8157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 -- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ed81f3180 msgr2=0x7f9ed8157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ed81f3180 0x7f9ed8157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 -- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ed80cec60 msgr2=0x7f9ed80cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ed80cec60 0x7f9ed80cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 -- 172.21.15.31:0/2911916414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed81f3c90 con 0x7f9ec0003970 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edd7b1640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ed80b8b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0037040 tx=0x7f9ed0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9ed5ffb640 1 -- 172.21.15.31:0/2911916414 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0003810 con 0x7f9ec0003970 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed81f3f70 con 0x7f9ec0003970 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9ed5ffb640 1 -- 172.21.15.31:0/2911916414 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0003970 con 0x7f9ec0003970 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.764+0000 7f9ed5ffb640 1 -- 172.21.15.31:0/2911916414 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0003d20 con 0x7f9ec0003970 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.765+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed81f4250 con 0x7f9ec0003970 2024-01-19T04:55:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.765+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed80cf650 con 0x7f9ec0003970 2024-01-19T04:55:23.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.766+0000 7f9ed5ffb640 1 -- 172.21.15.31:0/2911916414 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9ed0034440 con 0x7f9ec0003970 2024-01-19T04:55:23.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.766+0000 7f9ed5ffb640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ec406aed0 0x7f9ec406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:23.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.766+0000 7f9ed5ffb640 1 -- 172.21.15.31:0/2911916414 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ed00ab600 con 0x7f9ec0003970 2024-01-19T04:55:23.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.767+0000 7f9edcfb0640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ec406aed0 0x7f9ec406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:23.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.767+0000 7f9edcfb0640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ec406aed0 0x7f9ec406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ed815fca0 tx=0x7f9ecc007040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:23.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.771+0000 7f9ed5ffb640 1 -- 172.21.15.31:0/2911916414 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9ed80cf650 con 0x7f9ec0003970 2024-01-19T04:55:23.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.945+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ed81f4720 con 0x7f9ec406aed0 2024-01-19T04:55:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.952+0000 7f9ed5ffb640 1 -- 172.21.15.31:0/2911916414 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264687 (secure 0 0 0) 0x7f9ecc0119d0 con 0x7f9ec406aed0 2024-01-19T04:55:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:55:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ec406aed0 msgr2=0x7f9ec406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ec406aed0 0x7f9ec406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ed815fca0 tx=0x7f9ecc007040 comp rx=0 tx=0).stop 2024-01-19T04:55:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 msgr2=0x7f9ed80b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ed80b8b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0037040 tx=0x7f9ed0004730 comp rx=0 tx=0).stop 2024-01-19T04:55:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 shutdown_connections 2024-01-19T04:55:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ec406aed0 0x7f9ec406d390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ed80cec60 0x7f9ed80cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ed81f3180 0x7f9ed8157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 --2- 172.21.15.31:0/2911916414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec0003970 0x7f9ed80b8b50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 >> 172.21.15.31:0/2911916414 conn(0x7f9ed809c600 msgr2=0x7f9ed815b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 shutdown_connections 2024-01-19T04:55:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:23.955+0000 7f9edfa15640 1 -- 172.21.15.31:0/2911916414 wait complete. 2024-01-19T04:55:23.965 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-01-19T04:55:23.562112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219020,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755700,"statfs":{"total":773094113280,"available":772869836800,"internally_reserved":0,"allocated":2879488,"data_stored":1746536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671430+0000","last_change":"2024-01-19T04:55:19.508084+0000","last_active":"2024-01-19T04:55:20.671430+0000","last_peered":"2024-01-19T04:55:20.671430+0000","last_clean":"2024-01-19T04:55:20.671430+0000","last_became_active":"2024-01-19T04:55:19.507540+0000","last_became_peered":"2024-01-19T04:55:19.507540+0000","last_unstale":"2024-01-19T04:55:20.671430+0000","last_undegraded":"2024-01-19T04:55:20.671430+0000","last_fullsized":"2024-01-19T04:55:20.671430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:05.071465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491274+0000","last_change":"2024-01-19T04:55:19.503341+0000","last_active":"2024-01-19T04:55:20.491274+0000","last_peered":"2024-01-19T04:55:20.491274+0000","last_clean":"2024-01-19T04:55:20.491274+0000","last_became_active":"2024-01-19T04:55:19.503039+0000","last_became_peered":"2024-01-19T04:55:19.503039+0000","last_unstale":"2024-01-19T04:55:20.491274+0000","last_undegraded":"2024-01-19T04:55:20.491274+0000","last_fullsized":"2024-01-19T04:55:20.491274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825350+0000","last_change":"2024-01-19T04:55:19.498911+0000","last_active":"2024-01-19T04:55:20.825350+0000","last_peered":"2024-01-19T04:55:20.825350+0000","last_clean":"2024-01-19T04:55:20.825350+0000","last_became_active":"2024-01-19T04:55:19.498070+0000","last_became_peered":"2024-01-19T04:55:19.498070+0000","last_unstale":"2024-01-19T04:55:20.825350+0000","last_undegraded":"2024-01-19T04:55:20.825350+0000","last_fullsized":"2024-01-19T04:55:20.825350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492501+0000","last_change":"2024-01-19T04:55:19.501815+0000","last_active":"2024-01-19T04:55:20.492501+0000","last_peered":"2024-01-19T04:55:20.492501+0000","last_clean":"2024-01-19T04:55:20.492501+0000","last_became_active":"2024-01-19T04:55:19.501554+0000","last_became_peered":"2024-01-19T04:55:19.501554+0000","last_unstale":"2024-01-19T04:55:20.492501+0000","last_undegraded":"2024-01-19T04:55:20.492501+0000","last_fullsized":"2024-01-19T04:55:20.492501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508864+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.507962+0000","last_became_peered":"2024-01-19T04:55:19.507962+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492193+0000","last_change":"2024-01-19T04:55:19.504292+0000","last_active":"2024-01-19T04:55:20.492193+0000","last_peered":"2024-01-19T04:55:20.492193+0000","last_clean":"2024-01-19T04:55:20.492193+0000","last_became_active":"2024-01-19T04:55:19.504021+0000","last_became_peered":"2024-01-19T04:55:19.504021+0000","last_unstale":"2024-01-19T04:55:20.492193+0000","last_undegraded":"2024-01-19T04:55:20.492193+0000","last_fullsized":"2024-01-19T04:55:20.492193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670963+0000","last_change":"2024-01-19T04:55:19.507244+0000","last_active":"2024-01-19T04:55:20.670963+0000","last_peered":"2024-01-19T04:55:20.670963+0000","last_clean":"2024-01-19T04:55:20.670963+0000","last_became_active":"2024-01-19T04:55:19.506995+0000","last_became_peered":"2024-01-19T04:55:19.506995+0000","last_unstale":"2024-01-19T04:55:20.670963+0000","last_undegraded":"2024-01-19T04:55:20.670963+0000","last_fullsized":"2024-01-19T04:55:20.670963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973302+0000","last_change":"2024-01-19T04:55:19.505311+0000","last_active":"2024-01-19T04:55:20.973302+0000","last_peered":"2024-01-19T04:55:20.973302+0000","last_clean":"2024-01-19T04:55:20.973302+0000","last_became_active":"2024-01-19T04:55:19.504534+0000","last_became_peered":"2024-01-19T04:55:19.504534+0000","last_unstale":"2024-01-19T04:55:20.973302+0000","last_undegraded":"2024-01-19T04:55:20.973302+0000","last_fullsized":"2024-01-19T04:55:20.973302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671451+0000","last_change":"2024-01-19T04:55:19.508793+0000","last_active":"2024-01-19T04:55:20.671451+0000","last_peered":"2024-01-19T04:55:20.671451+0000","last_clean":"2024-01-19T04:55:20.671451+0000","last_became_active":"2024-01-19T04:55:19.508267+0000","last_became_peered":"2024-01-19T04:55:19.508267+0000","last_unstale":"2024-01-19T04:55:20.671451+0000","last_undegraded":"2024-01-19T04:55:20.671451+0000","last_fullsized":"2024-01-19T04:55:20.671451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:49:04.554504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973577+0000","last_change":"2024-01-19T04:55:19.506536+0000","last_active":"2024-01-19T04:55:20.973577+0000","last_peered":"2024-01-19T04:55:20.973577+0000","last_clean":"2024-01-19T04:55:20.973577+0000","last_became_active":"2024-01-19T04:55:19.506239+0000","last_became_peered":"2024-01-19T04:55:19.506239+0000","last_unstale":"2024-01-19T04:55:20.973577+0000","last_undegraded":"2024-01-19T04:55:20.973577+0000","last_fullsized":"2024-01-19T04:55:20.973577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973471+0000","last_change":"2024-01-19T04:55:19.502406+0000","last_active":"2024-01-19T04:55:20.973471+0000","last_peered":"2024-01-19T04:55:20.973471+0000","last_clean":"2024-01-19T04:55:20.973471+0000","last_became_active":"2024-01-19T04:55:19.502144+0000","last_became_peered":"2024-01-19T04:55:19.502144+0000","last_unstale":"2024-01-19T04:55:20.973471+0000","last_undegraded":"2024-01-19T04:55:20.973471+0000","last_fullsized":"2024-01-19T04:55:20.973471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493066+0000","last_change":"2024-01-19T04:55:19.503258+0000","last_active":"2024-01-19T04:55:20.493066+0000","last_peered":"2024-01-19T04:55:20.493066+0000","last_clean":"2024-01-19T04:55:20.493066+0000","last_became_active":"2024-01-19T04:55:19.503006+0000","last_became_peered":"2024-01-19T04:55:19.503006+0000","last_unstale":"2024-01-19T04:55:20.493066+0000","last_undegraded":"2024-01-19T04:55:20.493066+0000","last_fullsized":"2024-01-19T04:55:20.493066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973473+0000","last_change":"2024-01-19T04:55:19.507886+0000","last_active":"2024-01-19T04:55:20.973473+0000","last_peered":"2024-01-19T04:55:20.973473+0000","last_clean":"2024-01-19T04:55:20.973473+0000","last_became_active":"2024-01-19T04:55:19.507443+0000","last_became_peered":"2024-01-19T04:55:19.507443+0000","last_unstale":"2024-01-19T04:55:20.973473+0000","last_undegraded":"2024-01-19T04:55:20.973473+0000","last_fullsized":"2024-01-19T04:55:20.973473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825087+0000","last_change":"2024-01-19T04:55:19.502972+0000","last_active":"2024-01-19T04:55:20.825087+0000","last_peered":"2024-01-19T04:55:20.825087+0000","last_clean":"2024-01-19T04:55:20.825087+0000","last_became_active":"2024-01-19T04:55:19.502612+0000","last_became_peered":"2024-01-19T04:55:19.502612+0000","last_unstale":"2024-01-19T04:55:20.825087+0000","last_undegraded":"2024-01-19T04:55:20.825087+0000","last_fullsized":"2024-01-19T04:55:20.825087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:13.926376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491125+0000","last_change":"2024-01-19T04:55:19.503794+0000","last_active":"2024-01-19T04:55:20.491125+0000","last_peered":"2024-01-19T04:55:20.491125+0000","last_clean":"2024-01-19T04:55:20.491125+0000","last_became_active":"2024-01-19T04:55:19.503516+0000","last_became_peered":"2024-01-19T04:55:19.503516+0000","last_unstale":"2024-01-19T04:55:20.491125+0000","last_undegraded":"2024-01-19T04:55:20.491125+0000","last_fullsized":"2024-01-19T04:55:20.491125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491667+0000","last_change":"2024-01-19T04:55:19.500410+0000","last_active":"2024-01-19T04:55:20.491667+0000","last_peered":"2024-01-19T04:55:20.491667+0000","last_clean":"2024-01-19T04:55:20.491667+0000","last_became_active":"2024-01-19T04:55:19.500147+0000","last_became_peered":"2024-01-19T04:55:19.500147+0000","last_unstale":"2024-01-19T04:55:20.491667+0000","last_undegraded":"2024-01-19T04:55:20.491667+0000","last_fullsized":"2024-01-19T04:55:20.491667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493061+0000","last_change":"2024-01-19T04:55:19.507266+0000","last_active":"2024-01-19T04:55:20.493061+0000","last_peered":"2024-01-19T04:55:20.493061+0000","last_clean":"2024-01-19T04:55:20.493061+0000","last_became_active":"2024-01-19T04:55:19.506970+0000","last_became_peered":"2024-01-19T04:55:19.506970+0000","last_unstale":"2024-01-19T04:55:20.493061+0000","last_undegraded":"2024-01-19T04:55:20.493061+0000","last_fullsized":"2024-01-19T04:55:20.493061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973133+0000","last_change":"2024-01-19T04:55:19.501787+0000","last_active":"2024-01-19T04:55:20.973133+0000","last_peered":"2024-01-19T04:55:20.973133+0000","last_clean":"2024-01-19T04:55:20.973133+0000","last_became_active":"2024-01-19T04:55:19.501389+0000","last_became_peered":"2024-01-19T04:55:19.501389+0000","last_unstale":"2024-01-19T04:55:20.973133+0000","last_undegraded":"2024-01-19T04:55:20.973133+0000","last_fullsized":"2024-01-19T04:55:20.973133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671369+0000","last_change":"2024-01-19T04:55:19.507131+0000","last_active":"2024-01-19T04:55:20.671369+0000","last_peered":"2024-01-19T04:55:20.671369+0000","last_clean":"2024-01-19T04:55:20.671369+0000","last_became_active":"2024-01-19T04:55:19.506871+0000","last_became_peered":"2024-01-19T04:55:19.506871+0000","last_unstale":"2024-01-19T04:55:20.671369+0000","last_undegraded":"2024-01-19T04:55:20.671369+0000","last_fullsized":"2024-01-19T04:55:20.671369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:29:51.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973517+0000","last_change":"2024-01-19T04:55:19.504339+0000","last_active":"2024-01-19T04:55:20.973517+0000","last_peered":"2024-01-19T04:55:20.973517+0000","last_clean":"2024-01-19T04:55:20.973517+0000","last_became_active":"2024-01-19T04:55:19.504087+0000","last_became_peered":"2024-01-19T04:55:19.504087+0000","last_unstale":"2024-01-19T04:55:20.973517+0000","last_undegraded":"2024-01-19T04:55:20.973517+0000","last_fullsized":"2024-01-19T04:55:20.973517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825223+0000","last_change":"2024-01-19T04:55:19.510243+0000","last_active":"2024-01-19T04:55:20.825223+0000","last_peered":"2024-01-19T04:55:20.825223+0000","last_clean":"2024-01-19T04:55:20.825223+0000","last_became_active":"2024-01-19T04:55:19.509971+0000","last_became_peered":"2024-01-19T04:55:19.509971+0000","last_unstale":"2024-01-19T04:55:20.825223+0000","last_undegraded":"2024-01-19T04:55:20.825223+0000","last_fullsized":"2024-01-19T04:55:20.825223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492345+0000","last_change":"2024-01-19T04:55:19.509452+0000","last_active":"2024-01-19T04:55:20.492345+0000","last_peered":"2024-01-19T04:55:20.492345+0000","last_clean":"2024-01-19T04:55:20.492345+0000","last_became_active":"2024-01-19T04:55:19.507159+0000","last_became_peered":"2024-01-19T04:55:19.507159+0000","last_unstale":"2024-01-19T04:55:20.492345+0000","last_undegraded":"2024-01-19T04:55:20.492345+0000","last_fullsized":"2024-01-19T04:55:20.492345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973182+0000","last_change":"2024-01-19T04:55:19.508179+0000","last_active":"2024-01-19T04:55:20.973182+0000","last_peered":"2024-01-19T04:55:20.973182+0000","last_clean":"2024-01-19T04:55:20.973182+0000","last_became_active":"2024-01-19T04:55:19.507867+0000","last_became_peered":"2024-01-19T04:55:19.507867+0000","last_unstale":"2024-01-19T04:55:20.973182+0000","last_undegraded":"2024-01-19T04:55:20.973182+0000","last_fullsized":"2024-01-19T04:55:20.973182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491882+0000","last_change":"2024-01-19T04:55:19.500977+0000","last_active":"2024-01-19T04:55:20.491882+0000","last_peered":"2024-01-19T04:55:20.491882+0000","last_clean":"2024-01-19T04:55:20.491882+0000","last_became_active":"2024-01-19T04:55:19.500722+0000","last_became_peered":"2024-01-19T04:55:19.500722+0000","last_unstale":"2024-01-19T04:55:20.491882+0000","last_undegraded":"2024-01-19T04:55:20.491882+0000","last_fullsized":"2024-01-19T04:55:20.491882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973622+0000","last_change":"2024-01-19T04:55:19.505026+0000","last_active":"2024-01-19T04:55:20.973622+0000","last_peered":"2024-01-19T04:55:20.973622+0000","last_clean":"2024-01-19T04:55:20.973622+0000","last_became_active":"2024-01-19T04:55:19.504732+0000","last_became_peered":"2024-01-19T04:55:19.504732+0000","last_unstale":"2024-01-19T04:55:20.973622+0000","last_undegraded":"2024-01-19T04:55:20.973622+0000","last_fullsized":"2024-01-19T04:55:20.973622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973617+0000","last_change":"2024-01-19T04:55:19.508736+0000","last_active":"2024-01-19T04:55:20.973617+0000","last_peered":"2024-01-19T04:55:20.973617+0000","last_clean":"2024-01-19T04:55:20.973617+0000","last_became_active":"2024-01-19T04:55:19.508432+0000","last_became_peered":"2024-01-19T04:55:19.508432+0000","last_unstale":"2024-01-19T04:55:20.973617+0000","last_undegraded":"2024-01-19T04:55:20.973617+0000","last_fullsized":"2024-01-19T04:55:20.973617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491221+0000","last_change":"2024-01-19T04:55:19.508491+0000","last_active":"2024-01-19T04:55:20.491221+0000","last_peered":"2024-01-19T04:55:20.491221+0000","last_clean":"2024-01-19T04:55:20.491221+0000","last_became_active":"2024-01-19T04:55:19.508174+0000","last_became_peered":"2024-01-19T04:55:19.508174+0000","last_unstale":"2024-01-19T04:55:20.491221+0000","last_undegraded":"2024-01-19T04:55:20.491221+0000","last_fullsized":"2024-01-19T04:55:20.491221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973603+0000","last_change":"2024-01-19T04:55:19.506159+0000","last_active":"2024-01-19T04:55:20.973603+0000","last_peered":"2024-01-19T04:55:20.973603+0000","last_clean":"2024-01-19T04:55:20.973603+0000","last_became_active":"2024-01-19T04:55:19.505854+0000","last_became_peered":"2024-01-19T04:55:19.505854+0000","last_unstale":"2024-01-19T04:55:20.973603+0000","last_undegraded":"2024-01-19T04:55:20.973603+0000","last_fullsized":"2024-01-19T04:55:20.973603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825356+0000","last_change":"2024-01-19T04:55:19.509241+0000","last_active":"2024-01-19T04:55:20.825356+0000","last_peered":"2024-01-19T04:55:20.825356+0000","last_clean":"2024-01-19T04:55:20.825356+0000","last_became_active":"2024-01-19T04:55:19.508899+0000","last_became_peered":"2024-01-19T04:55:19.508899+0000","last_unstale":"2024-01-19T04:55:20.825356+0000","last_undegraded":"2024-01-19T04:55:20.825356+0000","last_fullsized":"2024-01-19T04:55:20.825356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492316+0000","last_change":"2024-01-19T04:55:19.505518+0000","last_active":"2024-01-19T04:55:20.492316+0000","last_peered":"2024-01-19T04:55:20.492316+0000","last_clean":"2024-01-19T04:55:20.492316+0000","last_became_active":"2024-01-19T04:55:19.505203+0000","last_became_peered":"2024-01-19T04:55:19.505203+0000","last_unstale":"2024-01-19T04:55:20.492316+0000","last_undegraded":"2024-01-19T04:55:20.492316+0000","last_fullsized":"2024-01-19T04:55:20.492316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973044+0000","last_change":"2024-01-19T04:55:19.500467+0000","last_active":"2024-01-19T04:55:20.973044+0000","last_peered":"2024-01-19T04:55:20.973044+0000","last_clean":"2024-01-19T04:55:20.973044+0000","last_became_active":"2024-01-19T04:55:19.500146+0000","last_became_peered":"2024-01-19T04:55:19.500146+0000","last_unstale":"2024-01-19T04:55:20.973044+0000","last_undegraded":"2024-01-19T04:55:20.973044+0000","last_fullsized":"2024-01-19T04:55:20.973044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973209+0000","last_change":"2024-01-19T04:55:19.504033+0000","last_active":"2024-01-19T04:55:20.973209+0000","last_peered":"2024-01-19T04:55:20.973209+0000","last_clean":"2024-01-19T04:55:20.973209+0000","last_became_active":"2024-01-19T04:55:19.503555+0000","last_became_peered":"2024-01-19T04:55:19.503555+0000","last_unstale":"2024-01-19T04:55:20.973209+0000","last_undegraded":"2024-01-19T04:55:20.973209+0000","last_fullsized":"2024-01-19T04:55:20.973209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492127+0000","last_change":"2024-01-19T04:55:19.505336+0000","last_active":"2024-01-19T04:55:20.492127+0000","last_peered":"2024-01-19T04:55:20.492127+0000","last_clean":"2024-01-19T04:55:20.492127+0000","last_became_active":"2024-01-19T04:55:19.505035+0000","last_became_peered":"2024-01-19T04:55:19.505035+0000","last_unstale":"2024-01-19T04:55:20.492127+0000","last_undegraded":"2024-01-19T04:55:20.492127+0000","last_fullsized":"2024-01-19T04:55:20.492127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:38.193951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973572+0000","last_change":"2024-01-19T04:55:18.533807+0000","last_active":"2024-01-19T04:55:20.973572+0000","last_peered":"2024-01-19T04:55:20.973572+0000","last_clean":"2024-01-19T04:55:20.973572+0000","last_became_active":"2024-01-19T04:55:18.533426+0000","last_became_peered":"2024-01-19T04:55:18.533426+0000","last_unstale":"2024-01-19T04:55:20.973572+0000","last_undegraded":"2024-01-19T04:55:20.973572+0000","last_fullsized":"2024-01-19T04:55:20.973572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825363+0000","last_change":"2024-01-19T04:55:19.502475+0000","last_active":"2024-01-19T04:55:20.825363+0000","last_peered":"2024-01-19T04:55:20.825363+0000","last_clean":"2024-01-19T04:55:20.825363+0000","last_became_active":"2024-01-19T04:55:19.502036+0000","last_became_peered":"2024-01-19T04:55:19.502036+0000","last_unstale":"2024-01-19T04:55:20.825363+0000","last_undegraded":"2024-01-19T04:55:20.825363+0000","last_fullsized":"2024-01-19T04:55:20.825363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973270+0000","last_change":"2024-01-19T04:55:18.525394+0000","last_active":"2024-01-19T04:55:20.973270+0000","last_peered":"2024-01-19T04:55:20.973270+0000","last_clean":"2024-01-19T04:55:20.973270+0000","last_became_active":"2024-01-19T04:55:18.525118+0000","last_became_peered":"2024-01-19T04:55:18.525118+0000","last_unstale":"2024-01-19T04:55:20.973270+0000","last_undegraded":"2024-01-19T04:55:20.973270+0000","last_fullsized":"2024-01-19T04:55:20.973270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973314+0000","last_change":"2024-01-19T04:55:18.526820+0000","last_active":"2024-01-19T04:55:20.973314+0000","last_peered":"2024-01-19T04:55:20.973314+0000","last_clean":"2024-01-19T04:55:20.973314+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.973314+0000","last_undegraded":"2024-01-19T04:55:20.973314+0000","last_fullsized":"2024-01-19T04:55:20.973314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973308+0000","last_change":"2024-01-19T04:55:19.505074+0000","last_active":"2024-01-19T04:55:20.973308+0000","last_peered":"2024-01-19T04:55:20.973308+0000","last_clean":"2024-01-19T04:55:20.973308+0000","last_became_active":"2024-01-19T04:55:19.504093+0000","last_became_peered":"2024-01-19T04:55:19.504093+0000","last_unstale":"2024-01-19T04:55:20.973308+0000","last_undegraded":"2024-01-19T04:55:20.973308+0000","last_fullsized":"2024-01-19T04:55:20.973308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491874+0000","last_change":"2024-01-19T04:55:18.520981+0000","last_active":"2024-01-19T04:55:20.491874+0000","last_peered":"2024-01-19T04:55:20.491874+0000","last_clean":"2024-01-19T04:55:20.491874+0000","last_became_active":"2024-01-19T04:55:18.520110+0000","last_became_peered":"2024-01-19T04:55:18.520110+0000","last_unstale":"2024-01-19T04:55:20.491874+0000","last_undegraded":"2024-01-19T04:55:20.491874+0000","last_fullsized":"2024-01-19T04:55:20.491874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491068+0000","last_change":"2024-01-19T04:55:19.510473+0000","last_active":"2024-01-19T04:55:20.491068+0000","last_peered":"2024-01-19T04:55:20.491068+0000","last_clean":"2024-01-19T04:55:20.491068+0000","last_became_active":"2024-01-19T04:55:19.510207+0000","last_became_peered":"2024-01-19T04:55:19.510207+0000","last_unstale":"2024-01-19T04:55:20.491068+0000","last_undegraded":"2024-01-19T04:55:20.491068+0000","last_fullsized":"2024-01-19T04:55:20.491068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973574+0000","last_change":"2024-01-19T04:55:18.528905+0000","last_active":"2024-01-19T04:55:20.973574+0000","last_peered":"2024-01-19T04:55:20.973574+0000","last_clean":"2024-01-19T04:55:20.973574+0000","last_became_active":"2024-01-19T04:55:18.528580+0000","last_became_peered":"2024-01-19T04:55:18.528580+0000","last_unstale":"2024-01-19T04:55:20.973574+0000","last_undegraded":"2024-01-19T04:55:20.973574+0000","last_fullsized":"2024-01-19T04:55:20.973574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492708+0000","last_change":"2024-01-19T04:55:19.509724+0000","last_active":"2024-01-19T04:55:20.492708+0000","last_peered":"2024-01-19T04:55:20.492708+0000","last_clean":"2024-01-19T04:55:20.492708+0000","last_became_active":"2024-01-19T04:55:19.509431+0000","last_became_peered":"2024-01-19T04:55:19.509431+0000","last_unstale":"2024-01-19T04:55:20.492708+0000","last_undegraded":"2024-01-19T04:55:20.492708+0000","last_fullsized":"2024-01-19T04:55:20.492708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492870+0000","last_change":"2024-01-19T04:55:18.525788+0000","last_active":"2024-01-19T04:55:20.492870+0000","last_peered":"2024-01-19T04:55:20.492870+0000","last_clean":"2024-01-19T04:55:20.492870+0000","last_became_active":"2024-01-19T04:55:18.525521+0000","last_became_peered":"2024-01-19T04:55:18.525521+0000","last_unstale":"2024-01-19T04:55:20.492870+0000","last_undegraded":"2024-01-19T04:55:20.492870+0000","last_fullsized":"2024-01-19T04:55:20.492870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825497+0000","last_change":"2024-01-19T04:55:19.501054+0000","last_active":"2024-01-19T04:55:20.825497+0000","last_peered":"2024-01-19T04:55:20.825497+0000","last_clean":"2024-01-19T04:55:20.825497+0000","last_became_active":"2024-01-19T04:55:19.500559+0000","last_became_peered":"2024-01-19T04:55:19.500559+0000","last_unstale":"2024-01-19T04:55:20.825497+0000","last_undegraded":"2024-01-19T04:55:20.825497+0000","last_fullsized":"2024-01-19T04:55:20.825497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491237+0000","last_change":"2024-01-19T04:55:18.528505+0000","last_active":"2024-01-19T04:55:20.491237+0000","last_peered":"2024-01-19T04:55:20.491237+0000","last_clean":"2024-01-19T04:55:20.491237+0000","last_became_active":"2024-01-19T04:55:18.528221+0000","last_became_peered":"2024-01-19T04:55:18.528221+0000","last_unstale":"2024-01-19T04:55:20.491237+0000","last_undegraded":"2024-01-19T04:55:20.491237+0000","last_fullsized":"2024-01-19T04:55:20.491237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:22.050835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493257+0000","last_change":"2024-01-19T04:55:19.507145+0000","last_active":"2024-01-19T04:55:20.493257+0000","last_peered":"2024-01-19T04:55:20.493257+0000","last_clean":"2024-01-19T04:55:20.493257+0000","last_became_active":"2024-01-19T04:55:19.506516+0000","last_became_peered":"2024-01-19T04:55:19.506516+0000","last_unstale":"2024-01-19T04:55:20.493257+0000","last_undegraded":"2024-01-19T04:55:20.493257+0000","last_fullsized":"2024-01-19T04:55:20.493257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:42.763778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973447+0000","last_change":"2024-01-19T04:55:18.527944+0000","last_active":"2024-01-19T04:55:20.973447+0000","last_peered":"2024-01-19T04:55:20.973447+0000","last_clean":"2024-01-19T04:55:20.973447+0000","last_became_active":"2024-01-19T04:55:18.527627+0000","last_became_peered":"2024-01-19T04:55:18.527627+0000","last_unstale":"2024-01-19T04:55:20.973447+0000","last_undegraded":"2024-01-19T04:55:20.973447+0000","last_fullsized":"2024-01-19T04:55:20.973447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508912+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.508650+0000","last_became_peered":"2024-01-19T04:55:19.508650+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825436+0000","last_change":"2024-01-19T04:55:18.529432+0000","last_active":"2024-01-19T04:55:20.825436+0000","last_peered":"2024-01-19T04:55:20.825436+0000","last_clean":"2024-01-19T04:55:20.825436+0000","last_became_active":"2024-01-19T04:55:18.529166+0000","last_became_peered":"2024-01-19T04:55:18.529166+0000","last_unstale":"2024-01-19T04:55:20.825436+0000","last_undegraded":"2024-01-19T04:55:20.825436+0000","last_fullsized":"2024-01-19T04:55:20.825436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825157+0000","last_change":"2024-01-19T04:55:19.507049+0000","last_active":"2024-01-19T04:55:20.825157+0000","last_peered":"2024-01-19T04:55:20.825157+0000","last_clean":"2024-01-19T04:55:20.825157+0000","last_became_active":"2024-01-19T04:55:19.506681+0000","last_became_peered":"2024-01-19T04:55:19.506681+0000","last_unstale":"2024-01-19T04:55:20.825157+0000","last_undegraded":"2024-01-19T04:55:20.825157+0000","last_fullsized":"2024-01-19T04:55:20.825157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492787+0000","last_change":"2024-01-19T04:55:18.522952+0000","last_active":"2024-01-19T04:55:20.492787+0000","last_peered":"2024-01-19T04:55:20.492787+0000","last_clean":"2024-01-19T04:55:20.492787+0000","last_became_active":"2024-01-19T04:55:18.522594+0000","last_became_peered":"2024-01-19T04:55:18.522594+0000","last_unstale":"2024-01-19T04:55:20.492787+0000","last_undegraded":"2024-01-19T04:55:20.492787+0000","last_fullsized":"2024-01-19T04:55:20.492787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671150+0000","last_change":"2024-01-19T04:55:19.509175+0000","last_active":"2024-01-19T04:55:20.671150+0000","last_peered":"2024-01-19T04:55:20.671150+0000","last_clean":"2024-01-19T04:55:20.671150+0000","last_became_active":"2024-01-19T04:55:19.508249+0000","last_became_peered":"2024-01-19T04:55:19.508249+0000","last_unstale":"2024-01-19T04:55:20.671150+0000","last_undegraded":"2024-01-19T04:55:20.671150+0000","last_fullsized":"2024-01-19T04:55:20.671150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973205+0000","last_change":"2024-01-19T04:55:18.525890+0000","last_active":"2024-01-19T04:55:20.973205+0000","last_peered":"2024-01-19T04:55:20.973205+0000","last_clean":"2024-01-19T04:55:20.973205+0000","last_became_active":"2024-01-19T04:55:18.525630+0000","last_became_peered":"2024-01-19T04:55:18.525630+0000","last_unstale":"2024-01-19T04:55:20.973205+0000","last_undegraded":"2024-01-19T04:55:20.973205+0000","last_fullsized":"2024-01-19T04:55:20.973205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973278+0000","last_change":"2024-01-19T04:55:19.503820+0000","last_active":"2024-01-19T04:55:20.973278+0000","last_peered":"2024-01-19T04:55:20.973278+0000","last_clean":"2024-01-19T04:55:20.973278+0000","last_became_active":"2024-01-19T04:55:19.503248+0000","last_became_peered":"2024-01-19T04:55:19.503248+0000","last_unstale":"2024-01-19T04:55:20.973278+0000","last_undegraded":"2024-01-19T04:55:20.973278+0000","last_fullsized":"2024-01-19T04:55:20.973278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973005+0000","last_change":"2024-01-19T04:55:18.521372+0000","last_active":"2024-01-19T04:55:20.973005+0000","last_peered":"2024-01-19T04:55:20.973005+0000","last_clean":"2024-01-19T04:55:20.973005+0000","last_became_active":"2024-01-19T04:55:18.521021+0000","last_became_peered":"2024-01-19T04:55:18.521021+0000","last_unstale":"2024-01-19T04:55:20.973005+0000","last_undegraded":"2024-01-19T04:55:20.973005+0000","last_fullsized":"2024-01-19T04:55:20.973005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671050+0000","last_change":"2024-01-19T04:55:19.507703+0000","last_active":"2024-01-19T04:55:20.671050+0000","last_peered":"2024-01-19T04:55:20.671050+0000","last_clean":"2024-01-19T04:55:20.671050+0000","last_became_active":"2024-01-19T04:55:19.507243+0000","last_became_peered":"2024-01-19T04:55:19.507243+0000","last_unstale":"2024-01-19T04:55:20.671050+0000","last_undegraded":"2024-01-19T04:55:20.671050+0000","last_fullsized":"2024-01-19T04:55:20.671050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492777+0000","last_change":"2024-01-19T04:55:18.531093+0000","last_active":"2024-01-19T04:55:20.492777+0000","last_peered":"2024-01-19T04:55:20.492777+0000","last_clean":"2024-01-19T04:55:20.492777+0000","last_became_active":"2024-01-19T04:55:18.530783+0000","last_became_peered":"2024-01-19T04:55:18.530783+0000","last_unstale":"2024-01-19T04:55:20.492777+0000","last_undegraded":"2024-01-19T04:55:20.492777+0000","last_fullsized":"2024-01-19T04:55:20.492777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:18:16.876671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671483+0000","last_change":"2024-01-19T04:55:19.506668+0000","last_active":"2024-01-19T04:55:20.671483+0000","last_peered":"2024-01-19T04:55:20.671483+0000","last_clean":"2024-01-19T04:55:20.671483+0000","last_became_active":"2024-01-19T04:55:19.506350+0000","last_became_peered":"2024-01-19T04:55:19.506350+0000","last_unstale":"2024-01-19T04:55:20.671483+0000","last_undegraded":"2024-01-19T04:55:20.671483+0000","last_fullsized":"2024-01-19T04:55:20.671483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:54:10.730504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973247+0000","last_change":"2024-01-19T04:55:18.523120+0000","last_active":"2024-01-19T04:55:20.973247+0000","last_peered":"2024-01-19T04:55:20.973247+0000","last_clean":"2024-01-19T04:55:20.973247+0000","last_became_active":"2024-01-19T04:55:18.522810+0000","last_became_peered":"2024-01-19T04:55:18.522810+0000","last_unstale":"2024-01-19T04:55:20.973247+0000","last_undegraded":"2024-01-19T04:55:20.973247+0000","last_fullsized":"2024-01-19T04:55:20.973247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671581+0000","last_change":"2024-01-19T04:55:19.508309+0000","last_active":"2024-01-19T04:55:20.671581+0000","last_peered":"2024-01-19T04:55:20.671581+0000","last_clean":"2024-01-19T04:55:20.671581+0000","last_became_active":"2024-01-19T04:55:19.507838+0000","last_became_peered":"2024-01-19T04:55:19.507838+0000","last_unstale":"2024-01-19T04:55:20.671581+0000","last_undegraded":"2024-01-19T04:55:20.671581+0000","last_fullsized":"2024-01-19T04:55:20.671581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:59:10.745652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491300+0000","last_change":"2024-01-19T04:55:18.526216+0000","last_active":"2024-01-19T04:55:20.491300+0000","last_peered":"2024-01-19T04:55:20.491300+0000","last_clean":"2024-01-19T04:55:20.491300+0000","last_became_active":"2024-01-19T04:55:18.524313+0000","last_became_peered":"2024-01-19T04:55:18.524313+0000","last_unstale":"2024-01-19T04:55:20.491300+0000","last_undegraded":"2024-01-19T04:55:20.491300+0000","last_fullsized":"2024-01-19T04:55:20.491300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492661+0000","last_change":"2024-01-19T04:55:19.501625+0000","last_active":"2024-01-19T04:55:20.492661+0000","last_peered":"2024-01-19T04:55:20.492661+0000","last_clean":"2024-01-19T04:55:20.492661+0000","last_became_active":"2024-01-19T04:55:19.500107+0000","last_became_peered":"2024-01-19T04:55:19.500107+0000","last_unstale":"2024-01-19T04:55:20.492661+0000","last_undegraded":"2024-01-19T04:55:20.492661+0000","last_fullsized":"2024-01-19T04:55:20.492661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973138+0000","last_change":"2024-01-19T04:55:18.525457+0000","last_active":"2024-01-19T04:55:20.973138+0000","last_peered":"2024-01-19T04:55:20.973138+0000","last_clean":"2024-01-19T04:55:20.973138+0000","last_became_active":"2024-01-19T04:55:18.525183+0000","last_became_peered":"2024-01-19T04:55:18.525183+0000","last_unstale":"2024-01-19T04:55:20.973138+0000","last_undegraded":"2024-01-19T04:55:20.973138+0000","last_fullsized":"2024-01-19T04:55:20.973138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671198+0000","last_change":"2024-01-19T04:55:19.508457+0000","last_active":"2024-01-19T04:55:20.671198+0000","last_peered":"2024-01-19T04:55:20.671198+0000","last_clean":"2024-01-19T04:55:20.671198+0000","last_became_active":"2024-01-19T04:55:19.506325+0000","last_became_peered":"2024-01-19T04:55:19.506325+0000","last_unstale":"2024-01-19T04:55:20.671198+0000","last_undegraded":"2024-01-19T04:55:20.671198+0000","last_fullsized":"2024-01-19T04:55:20.671198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:01:52.567337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-19T04:55:19.496795+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:55:19.496795+0000","last_peered":"2024-01-19T04:55:19.496795+0000","last_clean":"2024-01-19T04:55:19.496795+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:55:19.496795+0000","last_undegraded":"2024-01-19T04:55:19.496795+0000","last_fullsized":"2024-01-19T04:55:19.496795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492187+0000","last_change":"2024-01-19T04:55:18.520709+0000","last_active":"2024-01-19T04:55:20.492187+0000","last_peered":"2024-01-19T04:55:20.492187+0000","last_clean":"2024-01-19T04:55:20.492187+0000","last_became_active":"2024-01-19T04:55:18.520257+0000","last_became_peered":"2024-01-19T04:55:18.520257+0000","last_unstale":"2024-01-19T04:55:20.492187+0000","last_undegraded":"2024-01-19T04:55:20.492187+0000","last_fullsized":"2024-01-19T04:55:20.492187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973410+0000","last_change":"2024-01-19T04:55:19.507460+0000","last_active":"2024-01-19T04:55:20.973410+0000","last_peered":"2024-01-19T04:55:20.973410+0000","last_clean":"2024-01-19T04:55:20.973410+0000","last_became_active":"2024-01-19T04:55:19.507182+0000","last_became_peered":"2024-01-19T04:55:19.507182+0000","last_unstale":"2024-01-19T04:55:20.973410+0000","last_undegraded":"2024-01-19T04:55:20.973410+0000","last_fullsized":"2024-01-19T04:55:20.973410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670846+0000","last_change":"2024-01-19T04:55:18.531158+0000","last_active":"2024-01-19T04:55:20.670846+0000","last_peered":"2024-01-19T04:55:20.670846+0000","last_clean":"2024-01-19T04:55:20.670846+0000","last_became_active":"2024-01-19T04:55:18.530811+0000","last_became_peered":"2024-01-19T04:55:18.530811+0000","last_unstale":"2024-01-19T04:55:20.670846+0000","last_undegraded":"2024-01-19T04:55:20.670846+0000","last_fullsized":"2024-01-19T04:55:20.670846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:31.580264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825075+0000","last_change":"2024-01-19T04:55:19.503305+0000","last_active":"2024-01-19T04:55:20.825075+0000","last_peered":"2024-01-19T04:55:20.825075+0000","last_clean":"2024-01-19T04:55:20.825075+0000","last_became_active":"2024-01-19T04:55:19.502993+0000","last_became_peered":"2024-01-19T04:55:19.502993+0000","last_unstale":"2024-01-19T04:55:20.825075+0000","last_undegraded":"2024-01-19T04:55:20.825075+0000","last_fullsized":"2024-01-19T04:55:20.825075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:32.585555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973077+0000","last_change":"2024-01-19T04:55:18.523147+0000","last_active":"2024-01-19T04:55:20.973077+0000","last_peered":"2024-01-19T04:55:20.973077+0000","last_clean":"2024-01-19T04:55:20.973077+0000","last_became_active":"2024-01-19T04:55:18.522848+0000","last_became_peered":"2024-01-19T04:55:18.522848+0000","last_unstale":"2024-01-19T04:55:20.973077+0000","last_undegraded":"2024-01-19T04:55:20.973077+0000","last_fullsized":"2024-01-19T04:55:20.973077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491259+0000","last_change":"2024-01-19T04:55:19.500588+0000","last_active":"2024-01-19T04:55:20.491259+0000","last_peered":"2024-01-19T04:55:20.491259+0000","last_clean":"2024-01-19T04:55:20.491259+0000","last_became_active":"2024-01-19T04:55:19.500210+0000","last_became_peered":"2024-01-19T04:55:19.500210+0000","last_unstale":"2024-01-19T04:55:20.491259+0000","last_undegraded":"2024-01-19T04:55:20.491259+0000","last_fullsized":"2024-01-19T04:55:20.491259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492838+0000","last_change":"2024-01-19T04:55:18.521717+0000","last_active":"2024-01-19T04:55:20.492838+0000","last_peered":"2024-01-19T04:55:20.492838+0000","last_clean":"2024-01-19T04:55:20.492838+0000","last_became_active":"2024-01-19T04:55:18.521452+0000","last_became_peered":"2024-01-19T04:55:18.521452+0000","last_unstale":"2024-01-19T04:55:20.492838+0000","last_undegraded":"2024-01-19T04:55:20.492838+0000","last_fullsized":"2024-01-19T04:55:20.492838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492300+0000","last_change":"2024-01-19T04:55:19.497086+0000","last_active":"2024-01-19T04:55:20.492300+0000","last_peered":"2024-01-19T04:55:20.492300+0000","last_clean":"2024-01-19T04:55:20.492300+0000","last_became_active":"2024-01-19T04:55:19.496812+0000","last_became_peered":"2024-01-19T04:55:19.496812+0000","last_unstale":"2024-01-19T04:55:20.492300+0000","last_undegraded":"2024-01-19T04:55:20.492300+0000","last_fullsized":"2024-01-19T04:55:20.492300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:24.177458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491255+0000","last_change":"2024-01-19T04:55:18.531156+0000","last_active":"2024-01-19T04:55:20.491255+0000","last_peered":"2024-01-19T04:55:20.491255+0000","last_clean":"2024-01-19T04:55:20.491255+0000","last_became_active":"2024-01-19T04:55:18.530716+0000","last_became_peered":"2024-01-19T04:55:18.530716+0000","last_unstale":"2024-01-19T04:55:20.491255+0000","last_undegraded":"2024-01-19T04:55:20.491255+0000","last_fullsized":"2024-01-19T04:55:20.491255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492468+0000","last_change":"2024-01-19T04:55:19.497882+0000","last_active":"2024-01-19T04:55:20.492468+0000","last_peered":"2024-01-19T04:55:20.492468+0000","last_clean":"2024-01-19T04:55:20.492468+0000","last_became_active":"2024-01-19T04:55:19.497509+0000","last_became_peered":"2024-01-19T04:55:19.497509+0000","last_unstale":"2024-01-19T04:55:20.492468+0000","last_undegraded":"2024-01-19T04:55:20.492468+0000","last_fullsized":"2024-01-19T04:55:20.492468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671479+0000","last_change":"2024-01-19T04:55:18.539166+0000","last_active":"2024-01-19T04:55:20.671479+0000","last_peered":"2024-01-19T04:55:20.671479+0000","last_clean":"2024-01-19T04:55:20.671479+0000","last_became_active":"2024-01-19T04:55:18.538937+0000","last_became_peered":"2024-01-19T04:55:18.538937+0000","last_unstale":"2024-01-19T04:55:20.671479+0000","last_undegraded":"2024-01-19T04:55:20.671479+0000","last_fullsized":"2024-01-19T04:55:20.671479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:21:12.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491285+0000","last_change":"2024-01-19T04:55:19.499782+0000","last_active":"2024-01-19T04:55:20.491285+0000","last_peered":"2024-01-19T04:55:20.491285+0000","last_clean":"2024-01-19T04:55:20.491285+0000","last_became_active":"2024-01-19T04:55:19.499495+0000","last_became_peered":"2024-01-19T04:55:19.499495+0000","last_unstale":"2024-01-19T04:55:20.491285+0000","last_undegraded":"2024-01-19T04:55:20.491285+0000","last_fullsized":"2024-01-19T04:55:20.491285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491346+0000","last_change":"2024-01-19T04:55:18.530049+0000","last_active":"2024-01-19T04:55:20.491346+0000","last_peered":"2024-01-19T04:55:20.491346+0000","last_clean":"2024-01-19T04:55:20.491346+0000","last_became_active":"2024-01-19T04:55:18.529767+0000","last_became_peered":"2024-01-19T04:55:18.529767+0000","last_unstale":"2024-01-19T04:55:20.491346+0000","last_undegraded":"2024-01-19T04:55:20.491346+0000","last_fullsized":"2024-01-19T04:55:20.491346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973714+0000","last_change":"2024-01-19T04:55:19.511086+0000","last_active":"2024-01-19T04:55:20.973714+0000","last_peered":"2024-01-19T04:55:20.973714+0000","last_clean":"2024-01-19T04:55:20.973714+0000","last_became_active":"2024-01-19T04:55:19.510781+0000","last_became_peered":"2024-01-19T04:55:19.510781+0000","last_unstale":"2024-01-19T04:55:20.973714+0000","last_undegraded":"2024-01-19T04:55:20.973714+0000","last_fullsized":"2024-01-19T04:55:20.973714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973635+0000","last_change":"2024-01-19T04:55:18.529646+0000","last_active":"2024-01-19T04:55:20.973635+0000","last_peered":"2024-01-19T04:55:20.973635+0000","last_clean":"2024-01-19T04:55:20.973635+0000","last_became_active":"2024-01-19T04:55:18.529403+0000","last_became_peered":"2024-01-19T04:55:18.529403+0000","last_unstale":"2024-01-19T04:55:20.973635+0000","last_undegraded":"2024-01-19T04:55:20.973635+0000","last_fullsized":"2024-01-19T04:55:20.973635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825397+0000","last_change":"2024-01-19T04:55:19.503772+0000","last_active":"2024-01-19T04:55:20.825397+0000","last_peered":"2024-01-19T04:55:20.825397+0000","last_clean":"2024-01-19T04:55:20.825397+0000","last_became_active":"2024-01-19T04:55:19.503188+0000","last_became_peered":"2024-01-19T04:55:19.503188+0000","last_unstale":"2024-01-19T04:55:20.825397+0000","last_undegraded":"2024-01-19T04:55:20.825397+0000","last_fullsized":"2024-01-19T04:55:20.825397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670816+0000","last_change":"2024-01-19T04:55:18.536711+0000","last_active":"2024-01-19T04:55:20.670816+0000","last_peered":"2024-01-19T04:55:20.670816+0000","last_clean":"2024-01-19T04:55:20.670816+0000","last_became_active":"2024-01-19T04:55:18.536483+0000","last_became_peered":"2024-01-19T04:55:18.536483+0000","last_unstale":"2024-01-19T04:55:20.670816+0000","last_undegraded":"2024-01-19T04:55:20.670816+0000","last_fullsized":"2024-01-19T04:55:20.670816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:12.921084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492909+0000","last_change":"2024-01-19T04:55:19.496953+0000","last_active":"2024-01-19T04:55:20.492909+0000","last_peered":"2024-01-19T04:55:20.492909+0000","last_clean":"2024-01-19T04:55:20.492909+0000","last_became_active":"2024-01-19T04:55:19.496452+0000","last_became_peered":"2024-01-19T04:55:19.496452+0000","last_unstale":"2024-01-19T04:55:20.492909+0000","last_undegraded":"2024-01-19T04:55:20.492909+0000","last_fullsized":"2024-01-19T04:55:20.492909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491153+0000","last_change":"2024-01-19T04:55:18.529231+0000","last_active":"2024-01-19T04:55:20.491153+0000","last_peered":"2024-01-19T04:55:20.491153+0000","last_clean":"2024-01-19T04:55:20.491153+0000","last_became_active":"2024-01-19T04:55:18.528957+0000","last_became_peered":"2024-01-19T04:55:18.528957+0000","last_unstale":"2024-01-19T04:55:20.491153+0000","last_undegraded":"2024-01-19T04:55:20.491153+0000","last_fullsized":"2024-01-19T04:55:20.491153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973545+0000","last_change":"2024-01-19T04:55:19.504814+0000","last_active":"2024-01-19T04:55:20.973545+0000","last_peered":"2024-01-19T04:55:20.973545+0000","last_clean":"2024-01-19T04:55:20.973545+0000","last_became_active":"2024-01-19T04:55:19.503588+0000","last_became_peered":"2024-01-19T04:55:19.503588+0000","last_unstale":"2024-01-19T04:55:20.973545+0000","last_undegraded":"2024-01-19T04:55:20.973545+0000","last_fullsized":"2024-01-19T04:55:20.973545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491129+0000","last_change":"2024-01-19T04:55:18.526393+0000","last_active":"2024-01-19T04:55:20.491129+0000","last_peered":"2024-01-19T04:55:20.491129+0000","last_clean":"2024-01-19T04:55:20.491129+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.491129+0000","last_undegraded":"2024-01-19T04:55:20.491129+0000","last_fullsized":"2024-01-19T04:55:20.491129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973518+0000","last_change":"2024-01-19T04:55:19.507971+0000","last_active":"2024-01-19T04:55:20.973518+0000","last_peered":"2024-01-19T04:55:20.973518+0000","last_clean":"2024-01-19T04:55:20.973518+0000","last_became_active":"2024-01-19T04:55:19.507303+0000","last_became_peered":"2024-01-19T04:55:19.507303+0000","last_unstale":"2024-01-19T04:55:20.973518+0000","last_undegraded":"2024-01-19T04:55:20.973518+0000","last_fullsized":"2024-01-19T04:55:20.973518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671371+0000","last_change":"2024-01-19T04:55:18.532050+0000","last_active":"2024-01-19T04:55:20.671371+0000","last_peered":"2024-01-19T04:55:20.671371+0000","last_clean":"2024-01-19T04:55:20.671371+0000","last_became_active":"2024-01-19T04:55:18.531639+0000","last_became_peered":"2024-01-19T04:55:18.531639+0000","last_unstale":"2024-01-19T04:55:20.671371+0000","last_undegraded":"2024-01-19T04:55:20.671371+0000","last_fullsized":"2024-01-19T04:55:20.671371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:17:18.531824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973276+0000","last_change":"2024-01-19T04:55:19.504174+0000","last_active":"2024-01-19T04:55:20.973276+0000","last_peered":"2024-01-19T04:55:20.973276+0000","last_clean":"2024-01-19T04:55:20.973276+0000","last_became_active":"2024-01-19T04:55:19.503547+0000","last_became_peered":"2024-01-19T04:55:19.503547+0000","last_unstale":"2024-01-19T04:55:20.973276+0000","last_undegraded":"2024-01-19T04:55:20.973276+0000","last_fullsized":"2024-01-19T04:55:20.973276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492450+0000","last_change":"2024-01-19T04:55:19.499280+0000","last_active":"2024-01-19T04:55:20.492450+0000","last_peered":"2024-01-19T04:55:20.492450+0000","last_clean":"2024-01-19T04:55:20.492450+0000","last_became_active":"2024-01-19T04:55:19.499013+0000","last_became_peered":"2024-01-19T04:55:19.499013+0000","last_unstale":"2024-01-19T04:55:20.492450+0000","last_undegraded":"2024-01-19T04:55:20.492450+0000","last_fullsized":"2024-01-19T04:55:20.492450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671111+0000","last_change":"2024-01-19T04:55:18.529039+0000","last_active":"2024-01-19T04:55:20.671111+0000","last_peered":"2024-01-19T04:55:20.671111+0000","last_clean":"2024-01-19T04:55:20.671111+0000","last_became_active":"2024-01-19T04:55:18.528779+0000","last_became_peered":"2024-01-19T04:55:18.528779+0000","last_unstale":"2024-01-19T04:55:20.671111+0000","last_undegraded":"2024-01-19T04:55:20.671111+0000","last_fullsized":"2024-01-19T04:55:20.671111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:18:53.418269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492561+0000","last_change":"2024-01-19T04:55:19.499057+0000","last_active":"2024-01-19T04:55:20.492561+0000","last_peered":"2024-01-19T04:55:20.492561+0000","last_clean":"2024-01-19T04:55:20.492561+0000","last_became_active":"2024-01-19T04:55:19.498786+0000","last_became_peered":"2024-01-19T04:55:19.498786+0000","last_unstale":"2024-01-19T04:55:20.492561+0000","last_undegraded":"2024-01-19T04:55:20.492561+0000","last_fullsized":"2024-01-19T04:55:20.492561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671370+0000","last_change":"2024-01-19T04:55:18.532007+0000","last_active":"2024-01-19T04:55:20.671370+0000","last_peered":"2024-01-19T04:55:20.671370+0000","last_clean":"2024-01-19T04:55:20.671370+0000","last_became_active":"2024-01-19T04:55:18.531693+0000","last_became_peered":"2024-01-19T04:55:18.531693+0000","last_unstale":"2024-01-19T04:55:20.671370+0000","last_undegraded":"2024-01-19T04:55:20.671370+0000","last_fullsized":"2024-01-19T04:55:20.671370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825366+0000","last_change":"2024-01-19T04:55:19.500839+0000","last_active":"2024-01-19T04:55:20.825366+0000","last_peered":"2024-01-19T04:55:20.825366+0000","last_clean":"2024-01-19T04:55:20.825366+0000","last_became_active":"2024-01-19T04:55:19.500206+0000","last_became_peered":"2024-01-19T04:55:19.500206+0000","last_unstale":"2024-01-19T04:55:20.825366+0000","last_undegraded":"2024-01-19T04:55:20.825366+0000","last_fullsized":"2024-01-19T04:55:20.825366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973811+0000","last_change":"2024-01-19T04:55:18.532935+0000","last_active":"2024-01-19T04:55:20.973811+0000","last_peered":"2024-01-19T04:55:20.973811+0000","last_clean":"2024-01-19T04:55:20.973811+0000","last_became_active":"2024-01-19T04:55:18.532664+0000","last_became_peered":"2024-01-19T04:55:18.532664+0000","last_unstale":"2024-01-19T04:55:20.973811+0000","last_undegraded":"2024-01-19T04:55:20.973811+0000","last_fullsized":"2024-01-19T04:55:20.973811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973712+0000","last_change":"2024-01-19T04:55:19.508271+0000","last_active":"2024-01-19T04:55:20.973712+0000","last_peered":"2024-01-19T04:55:20.973712+0000","last_clean":"2024-01-19T04:55:20.973712+0000","last_became_active":"2024-01-19T04:55:19.507496+0000","last_became_peered":"2024-01-19T04:55:19.507496+0000","last_unstale":"2024-01-19T04:55:20.973712+0000","last_undegraded":"2024-01-19T04:55:20.973712+0000","last_fullsized":"2024-01-19T04:55:20.973712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825450+0000","last_change":"2024-01-19T04:55:18.531059+0000","last_active":"2024-01-19T04:55:20.825450+0000","last_peered":"2024-01-19T04:55:20.825450+0000","last_clean":"2024-01-19T04:55:20.825450+0000","last_became_active":"2024-01-19T04:55:18.530752+0000","last_became_peered":"2024-01-19T04:55:18.530752+0000","last_unstale":"2024-01-19T04:55:20.825450+0000","last_undegraded":"2024-01-19T04:55:20.825450+0000","last_fullsized":"2024-01-19T04:55:20.825450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:04:18.004474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:55:26.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:55:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.083+0000 7f479c843640 1 Processor -- start 2024-01-19T04:55:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.084+0000 7f479c843640 1 -- start start 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.084+0000 7f479c843640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4798161ee0 0x7f47980b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.084+0000 7f479c843640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f47980cf150 0x7f47980cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.084+0000 7f479c843640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980d22b0 con 0x7f47980d3cc0 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.084+0000 7f479c843640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980d41c0 con 0x7f4798161ee0 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.084+0000 7f479c843640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980d1980 con 0x7f47980cf150 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f479659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4798161ee0 0x7f47980b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f4796d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f47980d3cc0 0x7f47980cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56850/0 (socket says 172.21.15.31:56850) 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f4796d9d640 1 -- 172.21.15.31:0/3824041012 learned_addr learned my addr 172.21.15.31:0/3824041012 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f4795d9b640 1 --2- 172.21.15.31:0/3824041012 >> v2:172.21.15.137:3300/0 conn(0x7f47980cf150 0x7f47980cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f479659c640 1 -- 172.21.15.31:0/3824041012 >> v2:172.21.15.137:3300/0 conn(0x7f47980cf150 msgr2=0x7f47980cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f479659c640 1 --2- 172.21.15.31:0/3824041012 >> v2:172.21.15.137:3300/0 conn(0x7f47980cf150 0x7f47980cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f479659c640 1 -- 172.21.15.31:0/3824041012 >> v1:172.21.15.31:6789/0 conn(0x7f47980d3cc0 legacy=0x7f47980cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f479659c640 1 -- 172.21.15.31:0/3824041012 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980d1c60 con 0x7f4798161ee0 2024-01-19T04:55:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f479559a640 1 -- 172.21.15.31:0/3824041012 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661649666 0 0) 0x7f47980d22b0 con 0x7f47980d3cc0 2024-01-19T04:55:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.085+0000 7f4795d9b640 1 --2- 172.21.15.31:0/3824041012 >> v2:172.21.15.137:3300/0 conn(0x7f47980cf150 0x7f47980cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.086+0000 7f479659c640 1 --2- 172.21.15.31:0/3824041012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4798161ee0 0x7f47980b40d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f479806b230 tx=0x7f479002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5cad75df8e4753 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.086+0000 7f479559a640 1 -- 172.21.15.31:0/3824041012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790031830 con 0x7f4798161ee0 2024-01-19T04:55:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.086+0000 7f479559a640 1 -- 172.21.15.31:0/3824041012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790031990 con 0x7f4798161ee0 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.086+0000 7f479559a640 1 -- 172.21.15.31:0/3824041012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790031d20 con 0x7f4798161ee0 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.086+0000 7f479c843640 1 -- 172.21.15.31:0/3824041012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4798161ee0 msgr2=0x7f47980b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.086+0000 7f479c843640 1 --2- 172.21.15.31:0/3824041012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4798161ee0 0x7f47980b40d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f479806b230 tx=0x7f479002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.087+0000 7f479c843640 1 -- 172.21.15.31:0/3824041012 shutdown_connections 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.087+0000 7f479c843640 1 --2- 172.21.15.31:0/3824041012 >> v2:172.21.15.137:3300/0 conn(0x7f47980cf150 0x7f47980cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.087+0000 7f479c843640 1 --2- 172.21.15.31:0/3824041012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4798161ee0 0x7f47980b40d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.087+0000 7f479c843640 1 -- 172.21.15.31:0/3824041012 >> 172.21.15.31:0/3824041012 conn(0x7f479809c680 msgr2=0x7f47980d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.087+0000 7f479c843640 1 -- 172.21.15.31:0/3824041012 shutdown_connections 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.087+0000 7f479c843640 1 -- 172.21.15.31:0/3824041012 wait complete. 2024-01-19T04:55:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.088+0000 7f479c843640 1 Processor -- start 2024-01-19T04:55:27.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.088+0000 7f479c843640 1 -- start start 2024-01-19T04:55:27.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f479c843640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47980cf150 0x7f47981f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f479c843640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47980d3cc0 0x7f47981f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f479c843640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4798161ee0 0x7f4798155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f479c843640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4798156080 con 0x7f47980d3cc0 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f479c843640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47981f3b20 con 0x7f47980cf150 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f479c843640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4798155da0 con 0x7f4798161ee0 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47980d3cc0 0x7f47981f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47980d3cc0 0x7f47981f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36046/0 (socket says 172.21.15.31:36046) 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 -- 172.21.15.31:0/3538242118 learned_addr learned my addr 172.21.15.31:0/3538242118 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4796d9d640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4798161ee0 0x7f4798155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f479659c640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47980cf150 0x7f47981f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 -- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4798161ee0 msgr2=0x7f4798155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4798161ee0 0x7f4798155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 -- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47980cf150 msgr2=0x7f47981f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47980cf150 0x7f47981f64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 -- 172.21.15.31:0/3538242118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4790009400 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4796d9d640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4798161ee0 0x7f4798155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4795d9b640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47980d3cc0 0x7f47981f35e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f478c00e810 tx=0x7f478c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4786ffd640 1 -- 172.21.15.31:0/3538242118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c004230 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.089+0000 7f4786ffd640 1 -- 172.21.15.31:0/3538242118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c004390 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.090+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47980ca610 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.090+0000 7f4786ffd640 1 -- 172.21.15.31:0/3538242118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c00c910 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.090+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47980cabd0 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.091+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4798074a80 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.091+0000 7f4786ffd640 1 -- 172.21.15.31:0/3538242118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f478c00ca70 con 0x7f47980d3cc0 2024-01-19T04:55:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.091+0000 7f4786ffd640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f477806af00 0x7f477806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.092+0000 7f4786ffd640 1 -- 172.21.15.31:0/3538242118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f478c089250 con 0x7f47980d3cc0 2024-01-19T04:55:27.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.092+0000 7f479659c640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f477806af00 0x7f477806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.092+0000 7f479659c640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f477806af00 0x7f477806d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f479003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.096+0000 7f4786ffd640 1 -- 172.21.15.31:0/3538242118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f478c054090 con 0x7f47980d3cc0 2024-01-19T04:55:27.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.274+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4798067f50 con 0x7f477806af00 2024-01-19T04:55:27.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.281+0000 7f4786ffd640 1 -- 172.21.15.31:0/3538242118 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264687 (secure 0 0 0) 0x7f4798067f50 con 0x7f477806af00 2024-01-19T04:55:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:55:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f477806af00 msgr2=0x7f477806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f477806af00 0x7f477806d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f479003c040 comp rx=0 tx=0).stop 2024-01-19T04:55:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47980d3cc0 msgr2=0x7f47981f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47980d3cc0 0x7f47981f35e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f478c00e810 tx=0x7f478c00ecd0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 shutdown_connections 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f477806af00 0x7f477806d3c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4798161ee0 0x7f4798155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47980d3cc0 0x7f47981f35e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 --2- 172.21.15.31:0/3538242118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47980cf150 0x7f47981f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 >> 172.21.15.31:0/3538242118 conn(0x7f479809c680 msgr2=0x7f479806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 shutdown_connections 2024-01-19T04:55:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.284+0000 7f479c843640 1 -- 172.21.15.31:0/3538242118 wait complete. 2024-01-19T04:55:27.294 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-01-19T04:55:25.562709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219020,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755700,"statfs":{"total":773094113280,"available":772869836800,"internally_reserved":0,"allocated":2879488,"data_stored":1746536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.050270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671430+0000","last_change":"2024-01-19T04:55:19.508084+0000","last_active":"2024-01-19T04:55:20.671430+0000","last_peered":"2024-01-19T04:55:20.671430+0000","last_clean":"2024-01-19T04:55:20.671430+0000","last_became_active":"2024-01-19T04:55:19.507540+0000","last_became_peered":"2024-01-19T04:55:19.507540+0000","last_unstale":"2024-01-19T04:55:20.671430+0000","last_undegraded":"2024-01-19T04:55:20.671430+0000","last_fullsized":"2024-01-19T04:55:20.671430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:05.071465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491274+0000","last_change":"2024-01-19T04:55:19.503341+0000","last_active":"2024-01-19T04:55:20.491274+0000","last_peered":"2024-01-19T04:55:20.491274+0000","last_clean":"2024-01-19T04:55:20.491274+0000","last_became_active":"2024-01-19T04:55:19.503039+0000","last_became_peered":"2024-01-19T04:55:19.503039+0000","last_unstale":"2024-01-19T04:55:20.491274+0000","last_undegraded":"2024-01-19T04:55:20.491274+0000","last_fullsized":"2024-01-19T04:55:20.491274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825350+0000","last_change":"2024-01-19T04:55:19.498911+0000","last_active":"2024-01-19T04:55:20.825350+0000","last_peered":"2024-01-19T04:55:20.825350+0000","last_clean":"2024-01-19T04:55:20.825350+0000","last_became_active":"2024-01-19T04:55:19.498070+0000","last_became_peered":"2024-01-19T04:55:19.498070+0000","last_unstale":"2024-01-19T04:55:20.825350+0000","last_undegraded":"2024-01-19T04:55:20.825350+0000","last_fullsized":"2024-01-19T04:55:20.825350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492501+0000","last_change":"2024-01-19T04:55:19.501815+0000","last_active":"2024-01-19T04:55:20.492501+0000","last_peered":"2024-01-19T04:55:20.492501+0000","last_clean":"2024-01-19T04:55:20.492501+0000","last_became_active":"2024-01-19T04:55:19.501554+0000","last_became_peered":"2024-01-19T04:55:19.501554+0000","last_unstale":"2024-01-19T04:55:20.492501+0000","last_undegraded":"2024-01-19T04:55:20.492501+0000","last_fullsized":"2024-01-19T04:55:20.492501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508864+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.507962+0000","last_became_peered":"2024-01-19T04:55:19.507962+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492193+0000","last_change":"2024-01-19T04:55:19.504292+0000","last_active":"2024-01-19T04:55:20.492193+0000","last_peered":"2024-01-19T04:55:20.492193+0000","last_clean":"2024-01-19T04:55:20.492193+0000","last_became_active":"2024-01-19T04:55:19.504021+0000","last_became_peered":"2024-01-19T04:55:19.504021+0000","last_unstale":"2024-01-19T04:55:20.492193+0000","last_undegraded":"2024-01-19T04:55:20.492193+0000","last_fullsized":"2024-01-19T04:55:20.492193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670963+0000","last_change":"2024-01-19T04:55:19.507244+0000","last_active":"2024-01-19T04:55:20.670963+0000","last_peered":"2024-01-19T04:55:20.670963+0000","last_clean":"2024-01-19T04:55:20.670963+0000","last_became_active":"2024-01-19T04:55:19.506995+0000","last_became_peered":"2024-01-19T04:55:19.506995+0000","last_unstale":"2024-01-19T04:55:20.670963+0000","last_undegraded":"2024-01-19T04:55:20.670963+0000","last_fullsized":"2024-01-19T04:55:20.670963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973302+0000","last_change":"2024-01-19T04:55:19.505311+0000","last_active":"2024-01-19T04:55:20.973302+0000","last_peered":"2024-01-19T04:55:20.973302+0000","last_clean":"2024-01-19T04:55:20.973302+0000","last_became_active":"2024-01-19T04:55:19.504534+0000","last_became_peered":"2024-01-19T04:55:19.504534+0000","last_unstale":"2024-01-19T04:55:20.973302+0000","last_undegraded":"2024-01-19T04:55:20.973302+0000","last_fullsized":"2024-01-19T04:55:20.973302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671451+0000","last_change":"2024-01-19T04:55:19.508793+0000","last_active":"2024-01-19T04:55:20.671451+0000","last_peered":"2024-01-19T04:55:20.671451+0000","last_clean":"2024-01-19T04:55:20.671451+0000","last_became_active":"2024-01-19T04:55:19.508267+0000","last_became_peered":"2024-01-19T04:55:19.508267+0000","last_unstale":"2024-01-19T04:55:20.671451+0000","last_undegraded":"2024-01-19T04:55:20.671451+0000","last_fullsized":"2024-01-19T04:55:20.671451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:49:04.554504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973577+0000","last_change":"2024-01-19T04:55:19.506536+0000","last_active":"2024-01-19T04:55:20.973577+0000","last_peered":"2024-01-19T04:55:20.973577+0000","last_clean":"2024-01-19T04:55:20.973577+0000","last_became_active":"2024-01-19T04:55:19.506239+0000","last_became_peered":"2024-01-19T04:55:19.506239+0000","last_unstale":"2024-01-19T04:55:20.973577+0000","last_undegraded":"2024-01-19T04:55:20.973577+0000","last_fullsized":"2024-01-19T04:55:20.973577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973471+0000","last_change":"2024-01-19T04:55:19.502406+0000","last_active":"2024-01-19T04:55:20.973471+0000","last_peered":"2024-01-19T04:55:20.973471+0000","last_clean":"2024-01-19T04:55:20.973471+0000","last_became_active":"2024-01-19T04:55:19.502144+0000","last_became_peered":"2024-01-19T04:55:19.502144+0000","last_unstale":"2024-01-19T04:55:20.973471+0000","last_undegraded":"2024-01-19T04:55:20.973471+0000","last_fullsized":"2024-01-19T04:55:20.973471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493066+0000","last_change":"2024-01-19T04:55:19.503258+0000","last_active":"2024-01-19T04:55:20.493066+0000","last_peered":"2024-01-19T04:55:20.493066+0000","last_clean":"2024-01-19T04:55:20.493066+0000","last_became_active":"2024-01-19T04:55:19.503006+0000","last_became_peered":"2024-01-19T04:55:19.503006+0000","last_unstale":"2024-01-19T04:55:20.493066+0000","last_undegraded":"2024-01-19T04:55:20.493066+0000","last_fullsized":"2024-01-19T04:55:20.493066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973473+0000","last_change":"2024-01-19T04:55:19.507886+0000","last_active":"2024-01-19T04:55:20.973473+0000","last_peered":"2024-01-19T04:55:20.973473+0000","last_clean":"2024-01-19T04:55:20.973473+0000","last_became_active":"2024-01-19T04:55:19.507443+0000","last_became_peered":"2024-01-19T04:55:19.507443+0000","last_unstale":"2024-01-19T04:55:20.973473+0000","last_undegraded":"2024-01-19T04:55:20.973473+0000","last_fullsized":"2024-01-19T04:55:20.973473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825087+0000","last_change":"2024-01-19T04:55:19.502972+0000","last_active":"2024-01-19T04:55:20.825087+0000","last_peered":"2024-01-19T04:55:20.825087+0000","last_clean":"2024-01-19T04:55:20.825087+0000","last_became_active":"2024-01-19T04:55:19.502612+0000","last_became_peered":"2024-01-19T04:55:19.502612+0000","last_unstale":"2024-01-19T04:55:20.825087+0000","last_undegraded":"2024-01-19T04:55:20.825087+0000","last_fullsized":"2024-01-19T04:55:20.825087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:13.926376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491125+0000","last_change":"2024-01-19T04:55:19.503794+0000","last_active":"2024-01-19T04:55:20.491125+0000","last_peered":"2024-01-19T04:55:20.491125+0000","last_clean":"2024-01-19T04:55:20.491125+0000","last_became_active":"2024-01-19T04:55:19.503516+0000","last_became_peered":"2024-01-19T04:55:19.503516+0000","last_unstale":"2024-01-19T04:55:20.491125+0000","last_undegraded":"2024-01-19T04:55:20.491125+0000","last_fullsized":"2024-01-19T04:55:20.491125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491667+0000","last_change":"2024-01-19T04:55:19.500410+0000","last_active":"2024-01-19T04:55:20.491667+0000","last_peered":"2024-01-19T04:55:20.491667+0000","last_clean":"2024-01-19T04:55:20.491667+0000","last_became_active":"2024-01-19T04:55:19.500147+0000","last_became_peered":"2024-01-19T04:55:19.500147+0000","last_unstale":"2024-01-19T04:55:20.491667+0000","last_undegraded":"2024-01-19T04:55:20.491667+0000","last_fullsized":"2024-01-19T04:55:20.491667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493061+0000","last_change":"2024-01-19T04:55:19.507266+0000","last_active":"2024-01-19T04:55:20.493061+0000","last_peered":"2024-01-19T04:55:20.493061+0000","last_clean":"2024-01-19T04:55:20.493061+0000","last_became_active":"2024-01-19T04:55:19.506970+0000","last_became_peered":"2024-01-19T04:55:19.506970+0000","last_unstale":"2024-01-19T04:55:20.493061+0000","last_undegraded":"2024-01-19T04:55:20.493061+0000","last_fullsized":"2024-01-19T04:55:20.493061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973133+0000","last_change":"2024-01-19T04:55:19.501787+0000","last_active":"2024-01-19T04:55:20.973133+0000","last_peered":"2024-01-19T04:55:20.973133+0000","last_clean":"2024-01-19T04:55:20.973133+0000","last_became_active":"2024-01-19T04:55:19.501389+0000","last_became_peered":"2024-01-19T04:55:19.501389+0000","last_unstale":"2024-01-19T04:55:20.973133+0000","last_undegraded":"2024-01-19T04:55:20.973133+0000","last_fullsized":"2024-01-19T04:55:20.973133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671369+0000","last_change":"2024-01-19T04:55:19.507131+0000","last_active":"2024-01-19T04:55:20.671369+0000","last_peered":"2024-01-19T04:55:20.671369+0000","last_clean":"2024-01-19T04:55:20.671369+0000","last_became_active":"2024-01-19T04:55:19.506871+0000","last_became_peered":"2024-01-19T04:55:19.506871+0000","last_unstale":"2024-01-19T04:55:20.671369+0000","last_undegraded":"2024-01-19T04:55:20.671369+0000","last_fullsized":"2024-01-19T04:55:20.671369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:29:51.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973517+0000","last_change":"2024-01-19T04:55:19.504339+0000","last_active":"2024-01-19T04:55:20.973517+0000","last_peered":"2024-01-19T04:55:20.973517+0000","last_clean":"2024-01-19T04:55:20.973517+0000","last_became_active":"2024-01-19T04:55:19.504087+0000","last_became_peered":"2024-01-19T04:55:19.504087+0000","last_unstale":"2024-01-19T04:55:20.973517+0000","last_undegraded":"2024-01-19T04:55:20.973517+0000","last_fullsized":"2024-01-19T04:55:20.973517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825223+0000","last_change":"2024-01-19T04:55:19.510243+0000","last_active":"2024-01-19T04:55:20.825223+0000","last_peered":"2024-01-19T04:55:20.825223+0000","last_clean":"2024-01-19T04:55:20.825223+0000","last_became_active":"2024-01-19T04:55:19.509971+0000","last_became_peered":"2024-01-19T04:55:19.509971+0000","last_unstale":"2024-01-19T04:55:20.825223+0000","last_undegraded":"2024-01-19T04:55:20.825223+0000","last_fullsized":"2024-01-19T04:55:20.825223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492345+0000","last_change":"2024-01-19T04:55:19.509452+0000","last_active":"2024-01-19T04:55:20.492345+0000","last_peered":"2024-01-19T04:55:20.492345+0000","last_clean":"2024-01-19T04:55:20.492345+0000","last_became_active":"2024-01-19T04:55:19.507159+0000","last_became_peered":"2024-01-19T04:55:19.507159+0000","last_unstale":"2024-01-19T04:55:20.492345+0000","last_undegraded":"2024-01-19T04:55:20.492345+0000","last_fullsized":"2024-01-19T04:55:20.492345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973182+0000","last_change":"2024-01-19T04:55:19.508179+0000","last_active":"2024-01-19T04:55:20.973182+0000","last_peered":"2024-01-19T04:55:20.973182+0000","last_clean":"2024-01-19T04:55:20.973182+0000","last_became_active":"2024-01-19T04:55:19.507867+0000","last_became_peered":"2024-01-19T04:55:19.507867+0000","last_unstale":"2024-01-19T04:55:20.973182+0000","last_undegraded":"2024-01-19T04:55:20.973182+0000","last_fullsized":"2024-01-19T04:55:20.973182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491882+0000","last_change":"2024-01-19T04:55:19.500977+0000","last_active":"2024-01-19T04:55:20.491882+0000","last_peered":"2024-01-19T04:55:20.491882+0000","last_clean":"2024-01-19T04:55:20.491882+0000","last_became_active":"2024-01-19T04:55:19.500722+0000","last_became_peered":"2024-01-19T04:55:19.500722+0000","last_unstale":"2024-01-19T04:55:20.491882+0000","last_undegraded":"2024-01-19T04:55:20.491882+0000","last_fullsized":"2024-01-19T04:55:20.491882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973622+0000","last_change":"2024-01-19T04:55:19.505026+0000","last_active":"2024-01-19T04:55:20.973622+0000","last_peered":"2024-01-19T04:55:20.973622+0000","last_clean":"2024-01-19T04:55:20.973622+0000","last_became_active":"2024-01-19T04:55:19.504732+0000","last_became_peered":"2024-01-19T04:55:19.504732+0000","last_unstale":"2024-01-19T04:55:20.973622+0000","last_undegraded":"2024-01-19T04:55:20.973622+0000","last_fullsized":"2024-01-19T04:55:20.973622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973617+0000","last_change":"2024-01-19T04:55:19.508736+0000","last_active":"2024-01-19T04:55:20.973617+0000","last_peered":"2024-01-19T04:55:20.973617+0000","last_clean":"2024-01-19T04:55:20.973617+0000","last_became_active":"2024-01-19T04:55:19.508432+0000","last_became_peered":"2024-01-19T04:55:19.508432+0000","last_unstale":"2024-01-19T04:55:20.973617+0000","last_undegraded":"2024-01-19T04:55:20.973617+0000","last_fullsized":"2024-01-19T04:55:20.973617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491221+0000","last_change":"2024-01-19T04:55:19.508491+0000","last_active":"2024-01-19T04:55:20.491221+0000","last_peered":"2024-01-19T04:55:20.491221+0000","last_clean":"2024-01-19T04:55:20.491221+0000","last_became_active":"2024-01-19T04:55:19.508174+0000","last_became_peered":"2024-01-19T04:55:19.508174+0000","last_unstale":"2024-01-19T04:55:20.491221+0000","last_undegraded":"2024-01-19T04:55:20.491221+0000","last_fullsized":"2024-01-19T04:55:20.491221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973603+0000","last_change":"2024-01-19T04:55:19.506159+0000","last_active":"2024-01-19T04:55:20.973603+0000","last_peered":"2024-01-19T04:55:20.973603+0000","last_clean":"2024-01-19T04:55:20.973603+0000","last_became_active":"2024-01-19T04:55:19.505854+0000","last_became_peered":"2024-01-19T04:55:19.505854+0000","last_unstale":"2024-01-19T04:55:20.973603+0000","last_undegraded":"2024-01-19T04:55:20.973603+0000","last_fullsized":"2024-01-19T04:55:20.973603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825356+0000","last_change":"2024-01-19T04:55:19.509241+0000","last_active":"2024-01-19T04:55:20.825356+0000","last_peered":"2024-01-19T04:55:20.825356+0000","last_clean":"2024-01-19T04:55:20.825356+0000","last_became_active":"2024-01-19T04:55:19.508899+0000","last_became_peered":"2024-01-19T04:55:19.508899+0000","last_unstale":"2024-01-19T04:55:20.825356+0000","last_undegraded":"2024-01-19T04:55:20.825356+0000","last_fullsized":"2024-01-19T04:55:20.825356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492316+0000","last_change":"2024-01-19T04:55:19.505518+0000","last_active":"2024-01-19T04:55:20.492316+0000","last_peered":"2024-01-19T04:55:20.492316+0000","last_clean":"2024-01-19T04:55:20.492316+0000","last_became_active":"2024-01-19T04:55:19.505203+0000","last_became_peered":"2024-01-19T04:55:19.505203+0000","last_unstale":"2024-01-19T04:55:20.492316+0000","last_undegraded":"2024-01-19T04:55:20.492316+0000","last_fullsized":"2024-01-19T04:55:20.492316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973044+0000","last_change":"2024-01-19T04:55:19.500467+0000","last_active":"2024-01-19T04:55:20.973044+0000","last_peered":"2024-01-19T04:55:20.973044+0000","last_clean":"2024-01-19T04:55:20.973044+0000","last_became_active":"2024-01-19T04:55:19.500146+0000","last_became_peered":"2024-01-19T04:55:19.500146+0000","last_unstale":"2024-01-19T04:55:20.973044+0000","last_undegraded":"2024-01-19T04:55:20.973044+0000","last_fullsized":"2024-01-19T04:55:20.973044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973209+0000","last_change":"2024-01-19T04:55:19.504033+0000","last_active":"2024-01-19T04:55:20.973209+0000","last_peered":"2024-01-19T04:55:20.973209+0000","last_clean":"2024-01-19T04:55:20.973209+0000","last_became_active":"2024-01-19T04:55:19.503555+0000","last_became_peered":"2024-01-19T04:55:19.503555+0000","last_unstale":"2024-01-19T04:55:20.973209+0000","last_undegraded":"2024-01-19T04:55:20.973209+0000","last_fullsized":"2024-01-19T04:55:20.973209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492127+0000","last_change":"2024-01-19T04:55:19.505336+0000","last_active":"2024-01-19T04:55:20.492127+0000","last_peered":"2024-01-19T04:55:20.492127+0000","last_clean":"2024-01-19T04:55:20.492127+0000","last_became_active":"2024-01-19T04:55:19.505035+0000","last_became_peered":"2024-01-19T04:55:19.505035+0000","last_unstale":"2024-01-19T04:55:20.492127+0000","last_undegraded":"2024-01-19T04:55:20.492127+0000","last_fullsized":"2024-01-19T04:55:20.492127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:38.193951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973572+0000","last_change":"2024-01-19T04:55:18.533807+0000","last_active":"2024-01-19T04:55:20.973572+0000","last_peered":"2024-01-19T04:55:20.973572+0000","last_clean":"2024-01-19T04:55:20.973572+0000","last_became_active":"2024-01-19T04:55:18.533426+0000","last_became_peered":"2024-01-19T04:55:18.533426+0000","last_unstale":"2024-01-19T04:55:20.973572+0000","last_undegraded":"2024-01-19T04:55:20.973572+0000","last_fullsized":"2024-01-19T04:55:20.973572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825363+0000","last_change":"2024-01-19T04:55:19.502475+0000","last_active":"2024-01-19T04:55:20.825363+0000","last_peered":"2024-01-19T04:55:20.825363+0000","last_clean":"2024-01-19T04:55:20.825363+0000","last_became_active":"2024-01-19T04:55:19.502036+0000","last_became_peered":"2024-01-19T04:55:19.502036+0000","last_unstale":"2024-01-19T04:55:20.825363+0000","last_undegraded":"2024-01-19T04:55:20.825363+0000","last_fullsized":"2024-01-19T04:55:20.825363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973270+0000","last_change":"2024-01-19T04:55:18.525394+0000","last_active":"2024-01-19T04:55:20.973270+0000","last_peered":"2024-01-19T04:55:20.973270+0000","last_clean":"2024-01-19T04:55:20.973270+0000","last_became_active":"2024-01-19T04:55:18.525118+0000","last_became_peered":"2024-01-19T04:55:18.525118+0000","last_unstale":"2024-01-19T04:55:20.973270+0000","last_undegraded":"2024-01-19T04:55:20.973270+0000","last_fullsized":"2024-01-19T04:55:20.973270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973314+0000","last_change":"2024-01-19T04:55:18.526820+0000","last_active":"2024-01-19T04:55:20.973314+0000","last_peered":"2024-01-19T04:55:20.973314+0000","last_clean":"2024-01-19T04:55:20.973314+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.973314+0000","last_undegraded":"2024-01-19T04:55:20.973314+0000","last_fullsized":"2024-01-19T04:55:20.973314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973308+0000","last_change":"2024-01-19T04:55:19.505074+0000","last_active":"2024-01-19T04:55:20.973308+0000","last_peered":"2024-01-19T04:55:20.973308+0000","last_clean":"2024-01-19T04:55:20.973308+0000","last_became_active":"2024-01-19T04:55:19.504093+0000","last_became_peered":"2024-01-19T04:55:19.504093+0000","last_unstale":"2024-01-19T04:55:20.973308+0000","last_undegraded":"2024-01-19T04:55:20.973308+0000","last_fullsized":"2024-01-19T04:55:20.973308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491874+0000","last_change":"2024-01-19T04:55:18.520981+0000","last_active":"2024-01-19T04:55:20.491874+0000","last_peered":"2024-01-19T04:55:20.491874+0000","last_clean":"2024-01-19T04:55:20.491874+0000","last_became_active":"2024-01-19T04:55:18.520110+0000","last_became_peered":"2024-01-19T04:55:18.520110+0000","last_unstale":"2024-01-19T04:55:20.491874+0000","last_undegraded":"2024-01-19T04:55:20.491874+0000","last_fullsized":"2024-01-19T04:55:20.491874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491068+0000","last_change":"2024-01-19T04:55:19.510473+0000","last_active":"2024-01-19T04:55:20.491068+0000","last_peered":"2024-01-19T04:55:20.491068+0000","last_clean":"2024-01-19T04:55:20.491068+0000","last_became_active":"2024-01-19T04:55:19.510207+0000","last_became_peered":"2024-01-19T04:55:19.510207+0000","last_unstale":"2024-01-19T04:55:20.491068+0000","last_undegraded":"2024-01-19T04:55:20.491068+0000","last_fullsized":"2024-01-19T04:55:20.491068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973574+0000","last_change":"2024-01-19T04:55:18.528905+0000","last_active":"2024-01-19T04:55:20.973574+0000","last_peered":"2024-01-19T04:55:20.973574+0000","last_clean":"2024-01-19T04:55:20.973574+0000","last_became_active":"2024-01-19T04:55:18.528580+0000","last_became_peered":"2024-01-19T04:55:18.528580+0000","last_unstale":"2024-01-19T04:55:20.973574+0000","last_undegraded":"2024-01-19T04:55:20.973574+0000","last_fullsized":"2024-01-19T04:55:20.973574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492708+0000","last_change":"2024-01-19T04:55:19.509724+0000","last_active":"2024-01-19T04:55:20.492708+0000","last_peered":"2024-01-19T04:55:20.492708+0000","last_clean":"2024-01-19T04:55:20.492708+0000","last_became_active":"2024-01-19T04:55:19.509431+0000","last_became_peered":"2024-01-19T04:55:19.509431+0000","last_unstale":"2024-01-19T04:55:20.492708+0000","last_undegraded":"2024-01-19T04:55:20.492708+0000","last_fullsized":"2024-01-19T04:55:20.492708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492870+0000","last_change":"2024-01-19T04:55:18.525788+0000","last_active":"2024-01-19T04:55:20.492870+0000","last_peered":"2024-01-19T04:55:20.492870+0000","last_clean":"2024-01-19T04:55:20.492870+0000","last_became_active":"2024-01-19T04:55:18.525521+0000","last_became_peered":"2024-01-19T04:55:18.525521+0000","last_unstale":"2024-01-19T04:55:20.492870+0000","last_undegraded":"2024-01-19T04:55:20.492870+0000","last_fullsized":"2024-01-19T04:55:20.492870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825497+0000","last_change":"2024-01-19T04:55:19.501054+0000","last_active":"2024-01-19T04:55:20.825497+0000","last_peered":"2024-01-19T04:55:20.825497+0000","last_clean":"2024-01-19T04:55:20.825497+0000","last_became_active":"2024-01-19T04:55:19.500559+0000","last_became_peered":"2024-01-19T04:55:19.500559+0000","last_unstale":"2024-01-19T04:55:20.825497+0000","last_undegraded":"2024-01-19T04:55:20.825497+0000","last_fullsized":"2024-01-19T04:55:20.825497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491237+0000","last_change":"2024-01-19T04:55:18.528505+0000","last_active":"2024-01-19T04:55:20.491237+0000","last_peered":"2024-01-19T04:55:20.491237+0000","last_clean":"2024-01-19T04:55:20.491237+0000","last_became_active":"2024-01-19T04:55:18.528221+0000","last_became_peered":"2024-01-19T04:55:18.528221+0000","last_unstale":"2024-01-19T04:55:20.491237+0000","last_undegraded":"2024-01-19T04:55:20.491237+0000","last_fullsized":"2024-01-19T04:55:20.491237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:22.050835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493257+0000","last_change":"2024-01-19T04:55:19.507145+0000","last_active":"2024-01-19T04:55:20.493257+0000","last_peered":"2024-01-19T04:55:20.493257+0000","last_clean":"2024-01-19T04:55:20.493257+0000","last_became_active":"2024-01-19T04:55:19.506516+0000","last_became_peered":"2024-01-19T04:55:19.506516+0000","last_unstale":"2024-01-19T04:55:20.493257+0000","last_undegraded":"2024-01-19T04:55:20.493257+0000","last_fullsized":"2024-01-19T04:55:20.493257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:42.763778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973447+0000","last_change":"2024-01-19T04:55:18.527944+0000","last_active":"2024-01-19T04:55:20.973447+0000","last_peered":"2024-01-19T04:55:20.973447+0000","last_clean":"2024-01-19T04:55:20.973447+0000","last_became_active":"2024-01-19T04:55:18.527627+0000","last_became_peered":"2024-01-19T04:55:18.527627+0000","last_unstale":"2024-01-19T04:55:20.973447+0000","last_undegraded":"2024-01-19T04:55:20.973447+0000","last_fullsized":"2024-01-19T04:55:20.973447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508912+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.508650+0000","last_became_peered":"2024-01-19T04:55:19.508650+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825436+0000","last_change":"2024-01-19T04:55:18.529432+0000","last_active":"2024-01-19T04:55:20.825436+0000","last_peered":"2024-01-19T04:55:20.825436+0000","last_clean":"2024-01-19T04:55:20.825436+0000","last_became_active":"2024-01-19T04:55:18.529166+0000","last_became_peered":"2024-01-19T04:55:18.529166+0000","last_unstale":"2024-01-19T04:55:20.825436+0000","last_undegraded":"2024-01-19T04:55:20.825436+0000","last_fullsized":"2024-01-19T04:55:20.825436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825157+0000","last_change":"2024-01-19T04:55:19.507049+0000","last_active":"2024-01-19T04:55:20.825157+0000","last_peered":"2024-01-19T04:55:20.825157+0000","last_clean":"2024-01-19T04:55:20.825157+0000","last_became_active":"2024-01-19T04:55:19.506681+0000","last_became_peered":"2024-01-19T04:55:19.506681+0000","last_unstale":"2024-01-19T04:55:20.825157+0000","last_undegraded":"2024-01-19T04:55:20.825157+0000","last_fullsized":"2024-01-19T04:55:20.825157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492787+0000","last_change":"2024-01-19T04:55:18.522952+0000","last_active":"2024-01-19T04:55:20.492787+0000","last_peered":"2024-01-19T04:55:20.492787+0000","last_clean":"2024-01-19T04:55:20.492787+0000","last_became_active":"2024-01-19T04:55:18.522594+0000","last_became_peered":"2024-01-19T04:55:18.522594+0000","last_unstale":"2024-01-19T04:55:20.492787+0000","last_undegraded":"2024-01-19T04:55:20.492787+0000","last_fullsized":"2024-01-19T04:55:20.492787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671150+0000","last_change":"2024-01-19T04:55:19.509175+0000","last_active":"2024-01-19T04:55:20.671150+0000","last_peered":"2024-01-19T04:55:20.671150+0000","last_clean":"2024-01-19T04:55:20.671150+0000","last_became_active":"2024-01-19T04:55:19.508249+0000","last_became_peered":"2024-01-19T04:55:19.508249+0000","last_unstale":"2024-01-19T04:55:20.671150+0000","last_undegraded":"2024-01-19T04:55:20.671150+0000","last_fullsized":"2024-01-19T04:55:20.671150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973205+0000","last_change":"2024-01-19T04:55:18.525890+0000","last_active":"2024-01-19T04:55:20.973205+0000","last_peered":"2024-01-19T04:55:20.973205+0000","last_clean":"2024-01-19T04:55:20.973205+0000","last_became_active":"2024-01-19T04:55:18.525630+0000","last_became_peered":"2024-01-19T04:55:18.525630+0000","last_unstale":"2024-01-19T04:55:20.973205+0000","last_undegraded":"2024-01-19T04:55:20.973205+0000","last_fullsized":"2024-01-19T04:55:20.973205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973278+0000","last_change":"2024-01-19T04:55:19.503820+0000","last_active":"2024-01-19T04:55:20.973278+0000","last_peered":"2024-01-19T04:55:20.973278+0000","last_clean":"2024-01-19T04:55:20.973278+0000","last_became_active":"2024-01-19T04:55:19.503248+0000","last_became_peered":"2024-01-19T04:55:19.503248+0000","last_unstale":"2024-01-19T04:55:20.973278+0000","last_undegraded":"2024-01-19T04:55:20.973278+0000","last_fullsized":"2024-01-19T04:55:20.973278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973005+0000","last_change":"2024-01-19T04:55:18.521372+0000","last_active":"2024-01-19T04:55:20.973005+0000","last_peered":"2024-01-19T04:55:20.973005+0000","last_clean":"2024-01-19T04:55:20.973005+0000","last_became_active":"2024-01-19T04:55:18.521021+0000","last_became_peered":"2024-01-19T04:55:18.521021+0000","last_unstale":"2024-01-19T04:55:20.973005+0000","last_undegraded":"2024-01-19T04:55:20.973005+0000","last_fullsized":"2024-01-19T04:55:20.973005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671050+0000","last_change":"2024-01-19T04:55:19.507703+0000","last_active":"2024-01-19T04:55:20.671050+0000","last_peered":"2024-01-19T04:55:20.671050+0000","last_clean":"2024-01-19T04:55:20.671050+0000","last_became_active":"2024-01-19T04:55:19.507243+0000","last_became_peered":"2024-01-19T04:55:19.507243+0000","last_unstale":"2024-01-19T04:55:20.671050+0000","last_undegraded":"2024-01-19T04:55:20.671050+0000","last_fullsized":"2024-01-19T04:55:20.671050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492777+0000","last_change":"2024-01-19T04:55:18.531093+0000","last_active":"2024-01-19T04:55:20.492777+0000","last_peered":"2024-01-19T04:55:20.492777+0000","last_clean":"2024-01-19T04:55:20.492777+0000","last_became_active":"2024-01-19T04:55:18.530783+0000","last_became_peered":"2024-01-19T04:55:18.530783+0000","last_unstale":"2024-01-19T04:55:20.492777+0000","last_undegraded":"2024-01-19T04:55:20.492777+0000","last_fullsized":"2024-01-19T04:55:20.492777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:18:16.876671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671483+0000","last_change":"2024-01-19T04:55:19.506668+0000","last_active":"2024-01-19T04:55:20.671483+0000","last_peered":"2024-01-19T04:55:20.671483+0000","last_clean":"2024-01-19T04:55:20.671483+0000","last_became_active":"2024-01-19T04:55:19.506350+0000","last_became_peered":"2024-01-19T04:55:19.506350+0000","last_unstale":"2024-01-19T04:55:20.671483+0000","last_undegraded":"2024-01-19T04:55:20.671483+0000","last_fullsized":"2024-01-19T04:55:20.671483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:54:10.730504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973247+0000","last_change":"2024-01-19T04:55:18.523120+0000","last_active":"2024-01-19T04:55:20.973247+0000","last_peered":"2024-01-19T04:55:20.973247+0000","last_clean":"2024-01-19T04:55:20.973247+0000","last_became_active":"2024-01-19T04:55:18.522810+0000","last_became_peered":"2024-01-19T04:55:18.522810+0000","last_unstale":"2024-01-19T04:55:20.973247+0000","last_undegraded":"2024-01-19T04:55:20.973247+0000","last_fullsized":"2024-01-19T04:55:20.973247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671581+0000","last_change":"2024-01-19T04:55:19.508309+0000","last_active":"2024-01-19T04:55:20.671581+0000","last_peered":"2024-01-19T04:55:20.671581+0000","last_clean":"2024-01-19T04:55:20.671581+0000","last_became_active":"2024-01-19T04:55:19.507838+0000","last_became_peered":"2024-01-19T04:55:19.507838+0000","last_unstale":"2024-01-19T04:55:20.671581+0000","last_undegraded":"2024-01-19T04:55:20.671581+0000","last_fullsized":"2024-01-19T04:55:20.671581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:59:10.745652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491300+0000","last_change":"2024-01-19T04:55:18.526216+0000","last_active":"2024-01-19T04:55:20.491300+0000","last_peered":"2024-01-19T04:55:20.491300+0000","last_clean":"2024-01-19T04:55:20.491300+0000","last_became_active":"2024-01-19T04:55:18.524313+0000","last_became_peered":"2024-01-19T04:55:18.524313+0000","last_unstale":"2024-01-19T04:55:20.491300+0000","last_undegraded":"2024-01-19T04:55:20.491300+0000","last_fullsized":"2024-01-19T04:55:20.491300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492661+0000","last_change":"2024-01-19T04:55:19.501625+0000","last_active":"2024-01-19T04:55:20.492661+0000","last_peered":"2024-01-19T04:55:20.492661+0000","last_clean":"2024-01-19T04:55:20.492661+0000","last_became_active":"2024-01-19T04:55:19.500107+0000","last_became_peered":"2024-01-19T04:55:19.500107+0000","last_unstale":"2024-01-19T04:55:20.492661+0000","last_undegraded":"2024-01-19T04:55:20.492661+0000","last_fullsized":"2024-01-19T04:55:20.492661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973138+0000","last_change":"2024-01-19T04:55:18.525457+0000","last_active":"2024-01-19T04:55:20.973138+0000","last_peered":"2024-01-19T04:55:20.973138+0000","last_clean":"2024-01-19T04:55:20.973138+0000","last_became_active":"2024-01-19T04:55:18.525183+0000","last_became_peered":"2024-01-19T04:55:18.525183+0000","last_unstale":"2024-01-19T04:55:20.973138+0000","last_undegraded":"2024-01-19T04:55:20.973138+0000","last_fullsized":"2024-01-19T04:55:20.973138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671198+0000","last_change":"2024-01-19T04:55:19.508457+0000","last_active":"2024-01-19T04:55:20.671198+0000","last_peered":"2024-01-19T04:55:20.671198+0000","last_clean":"2024-01-19T04:55:20.671198+0000","last_became_active":"2024-01-19T04:55:19.506325+0000","last_became_peered":"2024-01-19T04:55:19.506325+0000","last_unstale":"2024-01-19T04:55:20.671198+0000","last_undegraded":"2024-01-19T04:55:20.671198+0000","last_fullsized":"2024-01-19T04:55:20.671198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:01:52.567337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-19T04:55:19.496795+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:55:19.496795+0000","last_peered":"2024-01-19T04:55:19.496795+0000","last_clean":"2024-01-19T04:55:19.496795+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:55:19.496795+0000","last_undegraded":"2024-01-19T04:55:19.496795+0000","last_fullsized":"2024-01-19T04:55:19.496795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492187+0000","last_change":"2024-01-19T04:55:18.520709+0000","last_active":"2024-01-19T04:55:20.492187+0000","last_peered":"2024-01-19T04:55:20.492187+0000","last_clean":"2024-01-19T04:55:20.492187+0000","last_became_active":"2024-01-19T04:55:18.520257+0000","last_became_peered":"2024-01-19T04:55:18.520257+0000","last_unstale":"2024-01-19T04:55:20.492187+0000","last_undegraded":"2024-01-19T04:55:20.492187+0000","last_fullsized":"2024-01-19T04:55:20.492187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973410+0000","last_change":"2024-01-19T04:55:19.507460+0000","last_active":"2024-01-19T04:55:20.973410+0000","last_peered":"2024-01-19T04:55:20.973410+0000","last_clean":"2024-01-19T04:55:20.973410+0000","last_became_active":"2024-01-19T04:55:19.507182+0000","last_became_peered":"2024-01-19T04:55:19.507182+0000","last_unstale":"2024-01-19T04:55:20.973410+0000","last_undegraded":"2024-01-19T04:55:20.973410+0000","last_fullsized":"2024-01-19T04:55:20.973410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670846+0000","last_change":"2024-01-19T04:55:18.531158+0000","last_active":"2024-01-19T04:55:20.670846+0000","last_peered":"2024-01-19T04:55:20.670846+0000","last_clean":"2024-01-19T04:55:20.670846+0000","last_became_active":"2024-01-19T04:55:18.530811+0000","last_became_peered":"2024-01-19T04:55:18.530811+0000","last_unstale":"2024-01-19T04:55:20.670846+0000","last_undegraded":"2024-01-19T04:55:20.670846+0000","last_fullsized":"2024-01-19T04:55:20.670846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:31.580264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825075+0000","last_change":"2024-01-19T04:55:19.503305+0000","last_active":"2024-01-19T04:55:20.825075+0000","last_peered":"2024-01-19T04:55:20.825075+0000","last_clean":"2024-01-19T04:55:20.825075+0000","last_became_active":"2024-01-19T04:55:19.502993+0000","last_became_peered":"2024-01-19T04:55:19.502993+0000","last_unstale":"2024-01-19T04:55:20.825075+0000","last_undegraded":"2024-01-19T04:55:20.825075+0000","last_fullsized":"2024-01-19T04:55:20.825075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:32.585555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973077+0000","last_change":"2024-01-19T04:55:18.523147+0000","last_active":"2024-01-19T04:55:20.973077+0000","last_peered":"2024-01-19T04:55:20.973077+0000","last_clean":"2024-01-19T04:55:20.973077+0000","last_became_active":"2024-01-19T04:55:18.522848+0000","last_became_peered":"2024-01-19T04:55:18.522848+0000","last_unstale":"2024-01-19T04:55:20.973077+0000","last_undegraded":"2024-01-19T04:55:20.973077+0000","last_fullsized":"2024-01-19T04:55:20.973077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491259+0000","last_change":"2024-01-19T04:55:19.500588+0000","last_active":"2024-01-19T04:55:20.491259+0000","last_peered":"2024-01-19T04:55:20.491259+0000","last_clean":"2024-01-19T04:55:20.491259+0000","last_became_active":"2024-01-19T04:55:19.500210+0000","last_became_peered":"2024-01-19T04:55:19.500210+0000","last_unstale":"2024-01-19T04:55:20.491259+0000","last_undegraded":"2024-01-19T04:55:20.491259+0000","last_fullsized":"2024-01-19T04:55:20.491259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492838+0000","last_change":"2024-01-19T04:55:18.521717+0000","last_active":"2024-01-19T04:55:20.492838+0000","last_peered":"2024-01-19T04:55:20.492838+0000","last_clean":"2024-01-19T04:55:20.492838+0000","last_became_active":"2024-01-19T04:55:18.521452+0000","last_became_peered":"2024-01-19T04:55:18.521452+0000","last_unstale":"2024-01-19T04:55:20.492838+0000","last_undegraded":"2024-01-19T04:55:20.492838+0000","last_fullsized":"2024-01-19T04:55:20.492838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492300+0000","last_change":"2024-01-19T04:55:19.497086+0000","last_active":"2024-01-19T04:55:20.492300+0000","last_peered":"2024-01-19T04:55:20.492300+0000","last_clean":"2024-01-19T04:55:20.492300+0000","last_became_active":"2024-01-19T04:55:19.496812+0000","last_became_peered":"2024-01-19T04:55:19.496812+0000","last_unstale":"2024-01-19T04:55:20.492300+0000","last_undegraded":"2024-01-19T04:55:20.492300+0000","last_fullsized":"2024-01-19T04:55:20.492300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:24.177458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491255+0000","last_change":"2024-01-19T04:55:18.531156+0000","last_active":"2024-01-19T04:55:20.491255+0000","last_peered":"2024-01-19T04:55:20.491255+0000","last_clean":"2024-01-19T04:55:20.491255+0000","last_became_active":"2024-01-19T04:55:18.530716+0000","last_became_peered":"2024-01-19T04:55:18.530716+0000","last_unstale":"2024-01-19T04:55:20.491255+0000","last_undegraded":"2024-01-19T04:55:20.491255+0000","last_fullsized":"2024-01-19T04:55:20.491255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492468+0000","last_change":"2024-01-19T04:55:19.497882+0000","last_active":"2024-01-19T04:55:20.492468+0000","last_peered":"2024-01-19T04:55:20.492468+0000","last_clean":"2024-01-19T04:55:20.492468+0000","last_became_active":"2024-01-19T04:55:19.497509+0000","last_became_peered":"2024-01-19T04:55:19.497509+0000","last_unstale":"2024-01-19T04:55:20.492468+0000","last_undegraded":"2024-01-19T04:55:20.492468+0000","last_fullsized":"2024-01-19T04:55:20.492468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671479+0000","last_change":"2024-01-19T04:55:18.539166+0000","last_active":"2024-01-19T04:55:20.671479+0000","last_peered":"2024-01-19T04:55:20.671479+0000","last_clean":"2024-01-19T04:55:20.671479+0000","last_became_active":"2024-01-19T04:55:18.538937+0000","last_became_peered":"2024-01-19T04:55:18.538937+0000","last_unstale":"2024-01-19T04:55:20.671479+0000","last_undegraded":"2024-01-19T04:55:20.671479+0000","last_fullsized":"2024-01-19T04:55:20.671479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:21:12.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491285+0000","last_change":"2024-01-19T04:55:19.499782+0000","last_active":"2024-01-19T04:55:20.491285+0000","last_peered":"2024-01-19T04:55:20.491285+0000","last_clean":"2024-01-19T04:55:20.491285+0000","last_became_active":"2024-01-19T04:55:19.499495+0000","last_became_peered":"2024-01-19T04:55:19.499495+0000","last_unstale":"2024-01-19T04:55:20.491285+0000","last_undegraded":"2024-01-19T04:55:20.491285+0000","last_fullsized":"2024-01-19T04:55:20.491285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491346+0000","last_change":"2024-01-19T04:55:18.530049+0000","last_active":"2024-01-19T04:55:20.491346+0000","last_peered":"2024-01-19T04:55:20.491346+0000","last_clean":"2024-01-19T04:55:20.491346+0000","last_became_active":"2024-01-19T04:55:18.529767+0000","last_became_peered":"2024-01-19T04:55:18.529767+0000","last_unstale":"2024-01-19T04:55:20.491346+0000","last_undegraded":"2024-01-19T04:55:20.491346+0000","last_fullsized":"2024-01-19T04:55:20.491346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973714+0000","last_change":"2024-01-19T04:55:19.511086+0000","last_active":"2024-01-19T04:55:20.973714+0000","last_peered":"2024-01-19T04:55:20.973714+0000","last_clean":"2024-01-19T04:55:20.973714+0000","last_became_active":"2024-01-19T04:55:19.510781+0000","last_became_peered":"2024-01-19T04:55:19.510781+0000","last_unstale":"2024-01-19T04:55:20.973714+0000","last_undegraded":"2024-01-19T04:55:20.973714+0000","last_fullsized":"2024-01-19T04:55:20.973714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973635+0000","last_change":"2024-01-19T04:55:18.529646+0000","last_active":"2024-01-19T04:55:20.973635+0000","last_peered":"2024-01-19T04:55:20.973635+0000","last_clean":"2024-01-19T04:55:20.973635+0000","last_became_active":"2024-01-19T04:55:18.529403+0000","last_became_peered":"2024-01-19T04:55:18.529403+0000","last_unstale":"2024-01-19T04:55:20.973635+0000","last_undegraded":"2024-01-19T04:55:20.973635+0000","last_fullsized":"2024-01-19T04:55:20.973635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825397+0000","last_change":"2024-01-19T04:55:19.503772+0000","last_active":"2024-01-19T04:55:20.825397+0000","last_peered":"2024-01-19T04:55:20.825397+0000","last_clean":"2024-01-19T04:55:20.825397+0000","last_became_active":"2024-01-19T04:55:19.503188+0000","last_became_peered":"2024-01-19T04:55:19.503188+0000","last_unstale":"2024-01-19T04:55:20.825397+0000","last_undegraded":"2024-01-19T04:55:20.825397+0000","last_fullsized":"2024-01-19T04:55:20.825397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670816+0000","last_change":"2024-01-19T04:55:18.536711+0000","last_active":"2024-01-19T04:55:20.670816+0000","last_peered":"2024-01-19T04:55:20.670816+0000","last_clean":"2024-01-19T04:55:20.670816+0000","last_became_active":"2024-01-19T04:55:18.536483+0000","last_became_peered":"2024-01-19T04:55:18.536483+0000","last_unstale":"2024-01-19T04:55:20.670816+0000","last_undegraded":"2024-01-19T04:55:20.670816+0000","last_fullsized":"2024-01-19T04:55:20.670816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:12.921084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492909+0000","last_change":"2024-01-19T04:55:19.496953+0000","last_active":"2024-01-19T04:55:20.492909+0000","last_peered":"2024-01-19T04:55:20.492909+0000","last_clean":"2024-01-19T04:55:20.492909+0000","last_became_active":"2024-01-19T04:55:19.496452+0000","last_became_peered":"2024-01-19T04:55:19.496452+0000","last_unstale":"2024-01-19T04:55:20.492909+0000","last_undegraded":"2024-01-19T04:55:20.492909+0000","last_fullsized":"2024-01-19T04:55:20.492909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491153+0000","last_change":"2024-01-19T04:55:18.529231+0000","last_active":"2024-01-19T04:55:20.491153+0000","last_peered":"2024-01-19T04:55:20.491153+0000","last_clean":"2024-01-19T04:55:20.491153+0000","last_became_active":"2024-01-19T04:55:18.528957+0000","last_became_peered":"2024-01-19T04:55:18.528957+0000","last_unstale":"2024-01-19T04:55:20.491153+0000","last_undegraded":"2024-01-19T04:55:20.491153+0000","last_fullsized":"2024-01-19T04:55:20.491153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973545+0000","last_change":"2024-01-19T04:55:19.504814+0000","last_active":"2024-01-19T04:55:20.973545+0000","last_peered":"2024-01-19T04:55:20.973545+0000","last_clean":"2024-01-19T04:55:20.973545+0000","last_became_active":"2024-01-19T04:55:19.503588+0000","last_became_peered":"2024-01-19T04:55:19.503588+0000","last_unstale":"2024-01-19T04:55:20.973545+0000","last_undegraded":"2024-01-19T04:55:20.973545+0000","last_fullsized":"2024-01-19T04:55:20.973545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491129+0000","last_change":"2024-01-19T04:55:18.526393+0000","last_active":"2024-01-19T04:55:20.491129+0000","last_peered":"2024-01-19T04:55:20.491129+0000","last_clean":"2024-01-19T04:55:20.491129+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.491129+0000","last_undegraded":"2024-01-19T04:55:20.491129+0000","last_fullsized":"2024-01-19T04:55:20.491129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973518+0000","last_change":"2024-01-19T04:55:19.507971+0000","last_active":"2024-01-19T04:55:20.973518+0000","last_peered":"2024-01-19T04:55:20.973518+0000","last_clean":"2024-01-19T04:55:20.973518+0000","last_became_active":"2024-01-19T04:55:19.507303+0000","last_became_peered":"2024-01-19T04:55:19.507303+0000","last_unstale":"2024-01-19T04:55:20.973518+0000","last_undegraded":"2024-01-19T04:55:20.973518+0000","last_fullsized":"2024-01-19T04:55:20.973518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671371+0000","last_change":"2024-01-19T04:55:18.532050+0000","last_active":"2024-01-19T04:55:20.671371+0000","last_peered":"2024-01-19T04:55:20.671371+0000","last_clean":"2024-01-19T04:55:20.671371+0000","last_became_active":"2024-01-19T04:55:18.531639+0000","last_became_peered":"2024-01-19T04:55:18.531639+0000","last_unstale":"2024-01-19T04:55:20.671371+0000","last_undegraded":"2024-01-19T04:55:20.671371+0000","last_fullsized":"2024-01-19T04:55:20.671371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:17:18.531824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973276+0000","last_change":"2024-01-19T04:55:19.504174+0000","last_active":"2024-01-19T04:55:20.973276+0000","last_peered":"2024-01-19T04:55:20.973276+0000","last_clean":"2024-01-19T04:55:20.973276+0000","last_became_active":"2024-01-19T04:55:19.503547+0000","last_became_peered":"2024-01-19T04:55:19.503547+0000","last_unstale":"2024-01-19T04:55:20.973276+0000","last_undegraded":"2024-01-19T04:55:20.973276+0000","last_fullsized":"2024-01-19T04:55:20.973276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492450+0000","last_change":"2024-01-19T04:55:19.499280+0000","last_active":"2024-01-19T04:55:20.492450+0000","last_peered":"2024-01-19T04:55:20.492450+0000","last_clean":"2024-01-19T04:55:20.492450+0000","last_became_active":"2024-01-19T04:55:19.499013+0000","last_became_peered":"2024-01-19T04:55:19.499013+0000","last_unstale":"2024-01-19T04:55:20.492450+0000","last_undegraded":"2024-01-19T04:55:20.492450+0000","last_fullsized":"2024-01-19T04:55:20.492450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671111+0000","last_change":"2024-01-19T04:55:18.529039+0000","last_active":"2024-01-19T04:55:20.671111+0000","last_peered":"2024-01-19T04:55:20.671111+0000","last_clean":"2024-01-19T04:55:20.671111+0000","last_became_active":"2024-01-19T04:55:18.528779+0000","last_became_peered":"2024-01-19T04:55:18.528779+0000","last_unstale":"2024-01-19T04:55:20.671111+0000","last_undegraded":"2024-01-19T04:55:20.671111+0000","last_fullsized":"2024-01-19T04:55:20.671111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:18:53.418269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492561+0000","last_change":"2024-01-19T04:55:19.499057+0000","last_active":"2024-01-19T04:55:20.492561+0000","last_peered":"2024-01-19T04:55:20.492561+0000","last_clean":"2024-01-19T04:55:20.492561+0000","last_became_active":"2024-01-19T04:55:19.498786+0000","last_became_peered":"2024-01-19T04:55:19.498786+0000","last_unstale":"2024-01-19T04:55:20.492561+0000","last_undegraded":"2024-01-19T04:55:20.492561+0000","last_fullsized":"2024-01-19T04:55:20.492561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671370+0000","last_change":"2024-01-19T04:55:18.532007+0000","last_active":"2024-01-19T04:55:20.671370+0000","last_peered":"2024-01-19T04:55:20.671370+0000","last_clean":"2024-01-19T04:55:20.671370+0000","last_became_active":"2024-01-19T04:55:18.531693+0000","last_became_peered":"2024-01-19T04:55:18.531693+0000","last_unstale":"2024-01-19T04:55:20.671370+0000","last_undegraded":"2024-01-19T04:55:20.671370+0000","last_fullsized":"2024-01-19T04:55:20.671370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825366+0000","last_change":"2024-01-19T04:55:19.500839+0000","last_active":"2024-01-19T04:55:20.825366+0000","last_peered":"2024-01-19T04:55:20.825366+0000","last_clean":"2024-01-19T04:55:20.825366+0000","last_became_active":"2024-01-19T04:55:19.500206+0000","last_became_peered":"2024-01-19T04:55:19.500206+0000","last_unstale":"2024-01-19T04:55:20.825366+0000","last_undegraded":"2024-01-19T04:55:20.825366+0000","last_fullsized":"2024-01-19T04:55:20.825366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973811+0000","last_change":"2024-01-19T04:55:18.532935+0000","last_active":"2024-01-19T04:55:20.973811+0000","last_peered":"2024-01-19T04:55:20.973811+0000","last_clean":"2024-01-19T04:55:20.973811+0000","last_became_active":"2024-01-19T04:55:18.532664+0000","last_became_peered":"2024-01-19T04:55:18.532664+0000","last_unstale":"2024-01-19T04:55:20.973811+0000","last_undegraded":"2024-01-19T04:55:20.973811+0000","last_fullsized":"2024-01-19T04:55:20.973811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973712+0000","last_change":"2024-01-19T04:55:19.508271+0000","last_active":"2024-01-19T04:55:20.973712+0000","last_peered":"2024-01-19T04:55:20.973712+0000","last_clean":"2024-01-19T04:55:20.973712+0000","last_became_active":"2024-01-19T04:55:19.507496+0000","last_became_peered":"2024-01-19T04:55:19.507496+0000","last_unstale":"2024-01-19T04:55:20.973712+0000","last_undegraded":"2024-01-19T04:55:20.973712+0000","last_fullsized":"2024-01-19T04:55:20.973712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825450+0000","last_change":"2024-01-19T04:55:18.531059+0000","last_active":"2024-01-19T04:55:20.825450+0000","last_peered":"2024-01-19T04:55:20.825450+0000","last_clean":"2024-01-19T04:55:20.825450+0000","last_became_active":"2024-01-19T04:55:18.530752+0000","last_became_peered":"2024-01-19T04:55:18.530752+0000","last_unstale":"2024-01-19T04:55:20.825450+0000","last_undegraded":"2024-01-19T04:55:20.825450+0000","last_fullsized":"2024-01-19T04:55:20.825450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:04:18.004474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:55:27.298 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-19T04:55:27.298 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-19T04:55:27.299 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-19T04:55:27.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-19T04:55:27.299 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-19T04:55:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.435+0000 7f4d7da58640 1 Processor -- start 2024-01-19T04:55:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.436+0000 7f4d7da58640 1 -- start start 2024-01-19T04:55:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.436+0000 7f4d7da58640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4d7806d6b0 0x7f4d780b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.436+0000 7f4d7da58640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d7815b260 0x7f4d7806da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.436+0000 7f4d7da58640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d780b2ee0 con 0x7f4d7813f9a0 2024-01-19T04:55:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.436+0000 7f4d7da58640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d780cba60 con 0x7f4d7815b260 2024-01-19T04:55:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.436+0000 7f4d7da58640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7815b6f0 con 0x7f4d7806d6b0 2024-01-19T04:55:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.436+0000 7f4d767fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d7815b260 0x7f4d7806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d767fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d7815b260 0x7f4d7806da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36062/0 (socket says 172.21.15.31:36062) 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d767fc640 1 -- 172.21.15.31:0/2450213472 learned_addr learned my addr 172.21.15.31:0/2450213472 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d76ffd640 1 --2- 172.21.15.31:0/2450213472 >> v2:172.21.15.137:3300/0 conn(0x7f4d7806d6b0 0x7f4d780b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d767fc640 1 -- 172.21.15.31:0/2450213472 >> v2:172.21.15.137:3300/0 conn(0x7f4d7806d6b0 msgr2=0x7f4d780b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d767fc640 1 --2- 172.21.15.31:0/2450213472 >> v2:172.21.15.137:3300/0 conn(0x7f4d7806d6b0 0x7f4d780b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d767fc640 1 -- 172.21.15.31:0/2450213472 >> v1:172.21.15.137:6789/0 conn(0x7f4d7813f9a0 legacy=0x7f4d7813fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d767fc640 1 -- 172.21.15.31:0/2450213472 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7806dfd0 con 0x7f4d7815b260 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.437+0000 7f4d767fc640 1 --2- 172.21.15.31:0/2450213472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7815b260 0x7f4d7806da90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d780b30c0 tx=0x7f4d6002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cdf6af26c62bd6 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d75ffb640 1 -- 172.21.15.31:0/2450213472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600317e0 con 0x7f4d7815b260 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d75ffb640 1 -- 172.21.15.31:0/2450213472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60031940 con 0x7f4d7815b260 2024-01-19T04:55:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d75ffb640 1 -- 172.21.15.31:0/2450213472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60031cd0 con 0x7f4d7815b260 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 -- 172.21.15.31:0/2450213472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7815b260 msgr2=0x7f4d7806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 --2- 172.21.15.31:0/2450213472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7815b260 0x7f4d7806da90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d780b30c0 tx=0x7f4d6002b880 comp rx=0 tx=0).stop 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 -- 172.21.15.31:0/2450213472 shutdown_connections 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 --2- 172.21.15.31:0/2450213472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7815b260 0x7f4d7806da90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 --2- 172.21.15.31:0/2450213472 >> v2:172.21.15.137:3300/0 conn(0x7f4d7806d6b0 0x7f4d780b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 -- 172.21.15.31:0/2450213472 >> 172.21.15.31:0/2450213472 conn(0x7f4d7806a790 msgr2=0x7f4d7806aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 -- 172.21.15.31:0/2450213472 shutdown_connections 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.438+0000 7f4d7da58640 1 -- 172.21.15.31:0/2450213472 wait complete. 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.439+0000 7f4d7da58640 1 Processor -- start 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d7da58640 1 -- start start 2024-01-19T04:55:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d7da58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7806d6b0 0x7f4d78143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d7da58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7813f9a0 0x7f4d7813cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d7da58640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4d7815b260 0x7f4d7813de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d7da58640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7813d110 con 0x7f4d7806d6b0 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d7da58640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780ce1c0 con 0x7f4d7813f9a0 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d7da58640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780cd1d0 con 0x7f4d7815b260 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d76ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7806d6b0 0x7f4d78143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d76ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7806d6b0 0x7f4d78143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36064/0 (socket says 172.21.15.31:36064) 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.440+0000 7f4d76ffd640 1 -- 172.21.15.31:0/4154843427 learned_addr learned my addr 172.21.15.31:0/4154843427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d777fe640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4d7815b260 0x7f4d7813de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d767fc640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7813f9a0 0x7f4d7813cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d76ffd640 1 -- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4d7815b260 msgr2=0x7f4d7813de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d76ffd640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4d7815b260 0x7f4d7813de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d76ffd640 1 -- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7813f9a0 msgr2=0x7f4d7813cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d76ffd640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7813f9a0 0x7f4d7813cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d76ffd640 1 -- 172.21.15.31:0/4154843427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600094d0 con 0x7f4d7806d6b0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d767fc640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7813f9a0 0x7f4d7813cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d76ffd640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7806d6b0 0x7f4d78143540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c7f0 tx=0x7f4d7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d6ffff640 1 -- 172.21.15.31:0/4154843427 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004220 con 0x7f4d7806d6b0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d6ffff640 1 -- 172.21.15.31:0/4154843427 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d700043a0 con 0x7f4d7806d6b0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d780cd4b0 con 0x7f4d7806d6b0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d6ffff640 1 -- 172.21.15.31:0/4154843427 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70007860 con 0x7f4d7806d6b0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.441+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d78142d40 con 0x7f4d7806d6b0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.442+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d780696e0 con 0x7f4d7806d6b0 2024-01-19T04:55:27.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.442+0000 7f4d6ffff640 1 -- 172.21.15.31:0/4154843427 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4d70007bd0 con 0x7f4d7806d6b0 2024-01-19T04:55:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.443+0000 7f4d6ffff640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d5806aed0 0x7f4d5806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.443+0000 7f4d6ffff640 1 -- 172.21.15.31:0/4154843427 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d70089660 con 0x7f4d7806d6b0 2024-01-19T04:55:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.443+0000 7f4d767fc640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d5806aed0 0x7f4d5806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.444+0000 7f4d767fc640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d5806aed0 0x7f4d5806d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d60036000 tx=0x7f4d6003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.447+0000 7f4d6ffff640 1 -- 172.21.15.31:0/4154843427 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4d70054530 con 0x7f4d7806d6b0 2024-01-19T04:55:27.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.678+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4d780ae8e0 con 0x7f4d7806d6b0 2024-01-19T04:55:27.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.681+0000 7f4d6ffff640 1 -- 172.21.15.31:0/4154843427 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170775 (secure 0 0 0) 0x7f4d70057b50 con 0x7f4d7806d6b0 2024-01-19T04:55:27.682 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:27.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d5806aed0 msgr2=0x7f4d5806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d5806aed0 0x7f4d5806d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d60036000 tx=0x7f4d6003d040 comp rx=0 tx=0).stop 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7806d6b0 msgr2=0x7f4d78143540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7806d6b0 0x7f4d78143540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c7f0 tx=0x7f4d7000ccb0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 shutdown_connections 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d5806aed0 0x7f4d5806d390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4d7815b260 0x7f4d7813de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7813f9a0 0x7f4d7813cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 --2- 172.21.15.31:0/4154843427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d7806d6b0 0x7f4d78143540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 >> 172.21.15.31:0/4154843427 conn(0x7f4d7806a790 msgr2=0x7f4d780aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 shutdown_connections 2024-01-19T04:55:27.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.683+0000 7f4d7da58640 1 -- 172.21.15.31:0/4154843427 wait complete. 2024-01-19T04:55:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4063332792},{"type":"v1","addr":"172.21.15.137:6833","nonce":4063332792}]},"active_addr":"172.21.15.137:6833/4063332792","active_change":"2024-01-19T04:54:59.549477+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":4119,"name":"y","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":2648443920}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":1357595767}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":3294555725}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":762796512}]}]} 2024-01-19T04:55:27.697 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-19T04:55:27.698 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-19T04:55:27.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.799+0000 7fe96557d640 1 Processor -- start 2024-01-19T04:55:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.799+0000 7fe96557d640 1 -- start start 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe96557d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9600cfd10 0x7fe9600d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe96557d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9600d0ef0 0x7fe9600d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe96557d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9600d06b0 con 0x7fe960160bb0 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe96557d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600d18c0 con 0x7fe9600cfd10 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe96557d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600d1b90 con 0x7fe9600d0ef0 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95e7fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9600cfd10 0x7fe9600d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95e7fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9600cfd10 0x7fe9600d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36074/0 (socket says 172.21.15.31:36074) 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95e7fc640 1 -- 172.21.15.31:0/1924663480 learned_addr learned my addr 172.21.15.31:0/1924663480 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95f7fe640 1 --2- 172.21.15.31:0/1924663480 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9600d0ef0 0x7fe9600d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95e7fc640 1 -- 172.21.15.31:0/1924663480 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9600d0ef0 msgr2=0x7fe9600d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95e7fc640 1 --2- 172.21.15.31:0/1924663480 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9600d0ef0 0x7fe9600d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95e7fc640 1 -- 172.21.15.31:0/1924663480 >> v1:172.21.15.137:6789/0 conn(0x7fe960160bb0 legacy=0x7fe9600b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.800+0000 7fe95e7fc640 1 -- 172.21.15.31:0/1924663480 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600d1e70 con 0x7fe9600cfd10 2024-01-19T04:55:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.801+0000 7fe95e7fc640 1 --2- 172.21.15.31:0/1924663480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9600d0170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe948009870 tx=0x7fe94802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5116da2a2279593 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.801+0000 7fe95dffb640 1 -- 172.21.15.31:0/1924663480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948031640 con 0x7fe9600cfd10 2024-01-19T04:55:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.801+0000 7fe95dffb640 1 -- 172.21.15.31:0/1924663480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9480317a0 con 0x7fe9600cfd10 2024-01-19T04:55:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.801+0000 7fe95dffb640 1 -- 172.21.15.31:0/1924663480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948031b30 con 0x7fe9600cfd10 2024-01-19T04:55:27.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 -- 172.21.15.31:0/1924663480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 msgr2=0x7fe9600d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 --2- 172.21.15.31:0/1924663480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9600d0170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe948009870 tx=0x7fe94802b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 -- 172.21.15.31:0/1924663480 shutdown_connections 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 --2- 172.21.15.31:0/1924663480 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9600d0ef0 0x7fe9600d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 --2- 172.21.15.31:0/1924663480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9600d0170 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 -- 172.21.15.31:0/1924663480 >> 172.21.15.31:0/1924663480 conn(0x7fe96009c600 msgr2=0x7fe9600d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 -- 172.21.15.31:0/1924663480 shutdown_connections 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.802+0000 7fe96557d640 1 -- 172.21.15.31:0/1924663480 wait complete. 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.803+0000 7fe96557d640 1 Processor -- start 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.803+0000 7fe96557d640 1 -- start start 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe96557d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9601599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe96557d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9600d0ef0 0x7fe9600b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe96557d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe960160bb0 0x7fe9600b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe96557d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960159ef0 con 0x7fe9600cfd10 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe96557d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600c7af0 con 0x7fe9600d0ef0 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe96557d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600c9280 con 0x7fe960160bb0 2024-01-19T04:55:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9601599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9601599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36076/0 (socket says 172.21.15.31:36076) 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 -- 172.21.15.31:0/1311587132 learned_addr learned my addr 172.21.15.31:0/1311587132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95f7fe640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe960160bb0 0x7fe9600b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 -- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe960160bb0 msgr2=0x7fe9600b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe960160bb0 0x7fe9600b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 -- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9600d0ef0 msgr2=0x7fe9600b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95e7fc640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9600d0ef0 0x7fe9600b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9600d0ef0 0x7fe9600b6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.804+0000 7fe95effd640 1 -- 172.21.15.31:0/1311587132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480094d0 con 0x7fe9600cfd10 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.805+0000 7fe95effd640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9601599b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b3f0 tx=0x7fe95800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.805+0000 7fe9577fe640 1 -- 172.21.15.31:0/1311587132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958011830 con 0x7fe9600cfd10 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.805+0000 7fe9577fe640 1 -- 172.21.15.31:0/1311587132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe958011990 con 0x7fe9600cfd10 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.805+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9600c9560 con 0x7fe9600cfd10 2024-01-19T04:55:27.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.805+0000 7fe9577fe640 1 -- 172.21.15.31:0/1311587132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958011d40 con 0x7fe9600cfd10 2024-01-19T04:55:27.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.805+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9600c9b20 con 0x7fe9600cfd10 2024-01-19T04:55:27.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.806+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9600cd6e0 con 0x7fe9600cfd10 2024-01-19T04:55:27.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.806+0000 7fe9577fe640 1 -- 172.21.15.31:0/1311587132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe95801a400 con 0x7fe9600cfd10 2024-01-19T04:55:27.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.806+0000 7fe9577fe640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe94006aed0 0x7fe94006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:27.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.807+0000 7fe9577fe640 1 -- 172.21.15.31:0/1311587132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9580894e0 con 0x7fe9600cfd10 2024-01-19T04:55:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.807+0000 7fe95e7fc640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe94006aed0 0x7fe94006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.807+0000 7fe95e7fc640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe94006aed0 0x7fe94006d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe948036000 tx=0x7fe94803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:27.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.811+0000 7fe9577fe640 1 -- 172.21.15.31:0/1311587132 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe9580543b0 con 0x7fe9600cfd10 2024-01-19T04:55:27.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.989+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9600696e0 con 0x7fe9600cfd10 2024-01-19T04:55:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.990+0000 7fe9577fe640 1 -- 172.21.15.31:0/1311587132 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fe9580579d0 con 0x7fe9600cfd10 2024-01-19T04:55:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:27.990 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe94006aed0 msgr2=0x7fe94006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe94006aed0 0x7fe94006d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe948036000 tx=0x7fe94803c040 comp rx=0 tx=0).stop 2024-01-19T04:55:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 msgr2=0x7fe9601599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9601599b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b3f0 tx=0x7fe95800b8b0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 shutdown_connections 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe94006aed0 0x7fe94006d390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe960160bb0 0x7fe9600b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9600d0ef0 0x7fe9600b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 --2- 172.21.15.31:0/1311587132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9600cfd10 0x7fe9601599b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 >> 172.21.15.31:0/1311587132 conn(0x7fe96009c600 msgr2=0x7fe96006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 shutdown_connections 2024-01-19T04:55:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:27.992+0000 7fe96557d640 1 -- 172.21.15.31:0/1311587132 wait complete. 2024-01-19T04:55:28.003 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-19T04:55:28.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:28.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.104+0000 7f52564ce640 1 Processor -- start 2024-01-19T04:55:28.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.104+0000 7f52564ce640 1 -- start start 2024-01-19T04:55:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.105+0000 7f52564ce640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5250160bb0 0x7f52500b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.105+0000 7f52564ce640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52500b66b0 con 0x7f52500cfd10 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.105+0000 7f52564ce640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52500d1b90 con 0x7f52500d1600 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.105+0000 7f52564ce640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52500d1e70 con 0x7f5250160bb0 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.105+0000 7f524f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52500cfd10 0x7f52500d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56858/0 (socket says 172.21.15.31:56858) 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.105+0000 7f524f7fe640 1 -- 172.21.15.31:0/1408764137 learned_addr learned my addr 172.21.15.31:0/1408764137 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524ffff640 1 --2- 172.21.15.31:0/1408764137 >> v2:172.21.15.137:3300/0 conn(0x7f5250160bb0 0x7f52500b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619567194 0 0) 0x7f52500b66b0 con 0x7f52500cfd10 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f523c003520 con 0x7f52500cfd10 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3699246750 0 0) 0x7f523c003520 con 0x7f52500cfd10 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 >> v2:172.21.15.137:3300/0 conn(0x7f5250160bb0 msgr2=0x7f52500b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 --2- 172.21.15.31:0/1408764137 >> v2:172.21.15.137:3300/0 conn(0x7f5250160bb0 0x7f52500b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 >> v1:172.21.15.137:6789/0 conn(0x7f52500d1600 legacy=0x7f52500ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500d2150 con 0x7f52500cfd10 2024-01-19T04:55:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f5238001670 con 0x7f52500cfd10 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 >> v1:172.21.15.31:6789/0 conn(0x7f52500cfd10 legacy=0x7f52500d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c003970 0x7f523c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f523c0047c0 0x7f523c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c005180 0x7f523c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500d1e70 con 0x7f523c0047c0 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c00c020 con 0x7f523c003970 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c00c1e0 con 0x7f523c005180 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524f7fe640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f523c0047c0 0x7f523c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524ffff640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c003970 0x7f523c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f5254a6b640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c005180 0x7f523c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524f7fe640 1 -- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c005180 msgr2=0x7f523c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524f7fe640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c005180 0x7f523c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524f7fe640 1 -- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c003970 msgr2=0x7f523c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524f7fe640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c003970 0x7f523c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f524f7fe640 1 -- 172.21.15.31:0/1408764137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c003520 con 0x7f523c0047c0 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.106+0000 7f5254a6b640 1 -- 172.21.15.31:0/1408764137 reap_dead start 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.107+0000 7f524f7fe640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f523c0047c0 0x7f523c004c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f523800c240 tx=0x7f523c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a3d0f7e34c1211 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.107+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238005ad0 con 0x7f523c0047c0 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.107+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238005c30 con 0x7f523c0047c0 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.107+0000 7f524effd640 1 -- 172.21.15.31:0/1408764137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523802cab0 con 0x7f523c0047c0 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.108+0000 7f52564ce640 1 -- 172.21.15.31:0/1408764137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f523c0047c0 msgr2=0x7f523c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.108+0000 7f52564ce640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f523c0047c0 0x7f523c004c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f523800c240 tx=0x7f523c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.108+0000 7f52564ce640 1 -- 172.21.15.31:0/1408764137 shutdown_connections 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.108+0000 7f52564ce640 1 --2- 172.21.15.31:0/1408764137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f523c0047c0 0x7f523c004c40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.108+0000 7f52564ce640 1 -- 172.21.15.31:0/1408764137 >> 172.21.15.31:0/1408764137 conn(0x7f525009c600 msgr2=0x7f52500d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.108+0000 7f52564ce640 1 -- 172.21.15.31:0/1408764137 shutdown_connections 2024-01-19T04:55:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.108+0000 7f52564ce640 1 -- 172.21.15.31:0/1408764137 wait complete. 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.109+0000 7f52564ce640 1 Processor -- start 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.109+0000 7f52564ce640 1 -- start start 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f52564ce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c0047c0 0x7f525014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f52564ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52500b7e00 0x7f52500b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f52564ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52500c85c0 0x7f52500c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f52564ce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525014fd30 con 0x7f52500b7e00 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f52564ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500b87c0 con 0x7f52500c85c0 2024-01-19T04:55:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f52564ce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525015a040 con 0x7f523c0047c0 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52500b7e00 0x7f52500b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52500b7e00 0x7f52500b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36098/0 (socket says 172.21.15.31:36098) 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 -- 172.21.15.31:0/4035309917 learned_addr learned my addr 172.21.15.31:0/4035309917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524ffff640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c0047c0 0x7f525014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f5254a6b640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52500c85c0 0x7f52500c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 -- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c0047c0 msgr2=0x7f525014f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c0047c0 0x7f525014f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 -- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52500c85c0 msgr2=0x7f52500c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52500c85c0 0x7f52500c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 -- 172.21.15.31:0/4035309917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500c9bb0 con 0x7f52500b7e00 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f5254a6b640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52500c85c0 0x7f52500c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524f7fe640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52500b7e00 0x7f52500b8280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5238001930 tx=0x7f523802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524cff9640 1 -- 172.21.15.31:0/4035309917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523803c040 con 0x7f52500b7e00 2024-01-19T04:55:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524cff9640 1 -- 172.21.15.31:0/4035309917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523802cc80 con 0x7f52500b7e00 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52500c9310 con 0x7f52500b7e00 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.110+0000 7f524cff9640 1 -- 172.21.15.31:0/4035309917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238007070 con 0x7f52500b7e00 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.111+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52500c98d0 con 0x7f52500b7e00 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.112+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52500696e0 con 0x7f52500b7e00 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.112+0000 7f524cff9640 1 -- 172.21.15.31:0/4035309917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5238032070 con 0x7f52500b7e00 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.112+0000 7f524cff9640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f523006af00 0x7f523006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.113+0000 7f524cff9640 1 -- 172.21.15.31:0/4035309917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52380aab00 con 0x7f52500b7e00 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.113+0000 7f524ffff640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f523006af00 0x7f523006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.113+0000 7f524ffff640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f523006af00 0x7f523006d3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5240009b00 tx=0x7f5240008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.117+0000 7f524cff9640 1 -- 172.21.15.31:0/4035309917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f52380759f0 con 0x7f52500b7e00 2024-01-19T04:55:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.294+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52500c9dd0 con 0x7f52500b7e00 2024-01-19T04:55:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.295+0000 7f524cff9640 1 -- 172.21.15.31:0/4035309917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f5238046070 con 0x7f52500b7e00 2024-01-19T04:55:28.295 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:28.295 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f523006af00 msgr2=0x7f523006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f523006af00 0x7f523006d3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5240009b00 tx=0x7f5240008040 comp rx=0 tx=0).stop 2024-01-19T04:55:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52500b7e00 msgr2=0x7f52500b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52500b7e00 0x7f52500b8280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5238001930 tx=0x7f523802e070 comp rx=0 tx=0).stop 2024-01-19T04:55:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 shutdown_connections 2024-01-19T04:55:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f523006af00 0x7f523006d3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52500c85c0 0x7f52500c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52500b7e00 0x7f52500b8280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 --2- 172.21.15.31:0/4035309917 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f523c0047c0 0x7f525014f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.297+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 >> 172.21.15.31:0/4035309917 conn(0x7f525009c600 msgr2=0x7f5250161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.298+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 shutdown_connections 2024-01-19T04:55:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.298+0000 7f52564ce640 1 -- 172.21.15.31:0/4035309917 wait complete. 2024-01-19T04:55:28.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-19T04:55:28.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-19T04:55:28.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-19T04:55:28.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-19T04:55:28.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-19T04:55:28.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-19T04:55:28.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-19T04:55:28.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-19T04:55:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.484+0000 7fb217989640 1 Processor -- start 2024-01-19T04:55:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.484+0000 7fb217989640 1 -- start start 2024-01-19T04:55:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.485+0000 7f767a79e640 1 Processor -- start 2024-01-19T04:55:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7f767a79e640 1 -- start start 2024-01-19T04:55:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7f2fad8ee640 1 Processor -- start 2024-01-19T04:55:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7fb217989640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb21014b340 0x7fb2100714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7fb217989640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb210071a10 0x7fb21006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7f8c8adcd640 1 Processor -- start 2024-01-19T04:55:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7fb217989640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21006aaf0 0x7fb210165460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7fb217989640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21006af80 con 0x7fb21014b340 2024-01-19T04:55:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7fb217989640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb210071e50 con 0x7fb210071a10 2024-01-19T04:55:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7fb217989640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21006e180 con 0x7fb21006aaf0 2024-01-19T04:55:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7f830915a640 1 Processor -- start 2024-01-19T04:55:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7f830915a640 1 -- start start 2024-01-19T04:55:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215f26640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21006aaf0 0x7fb210165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7efdbc741640 1 Processor -- start 2024-01-19T04:55:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.486+0000 7f2fad8ee640 1 -- start start 2024-01-19T04:55:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215725640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb21014b340 0x7fb2100714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7f2b1599e640 1 Processor -- start 2024-01-19T04:55:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215f26640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21006aaf0 0x7fb210165460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:60958/0 (socket says 172.21.15.31:60958) 2024-01-19T04:55:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215f26640 1 -- 172.21.15.31:0/3812630097 learned_addr learned my addr 172.21.15.31:0/3812630097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7efdbc741640 1 -- start start 2024-01-19T04:55:28.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215725640 1 -- 172.21.15.31:0/3812630097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21006aaf0 msgr2=0x7fb210165460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7f2b1599e640 1 -- start start 2024-01-19T04:55:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215725640 1 --2- 172.21.15.31:0/3812630097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21006aaf0 0x7fb210165460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7f2fad8ee640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2fa8071110 0x7f2fa806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215725640 1 -- 172.21.15.31:0/3812630097 >> v2:172.21.15.137:3300/0 conn(0x7fb210071a10 msgr2=0x7fb21006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f2fad8ee640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa806b430 con 0x7f2fa80cea50 2024-01-19T04:55:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f2fad8ee640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa806d8c0 con 0x7f2fa806bc90 2024-01-19T04:55:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215725640 1 --2- 172.21.15.31:0/3812630097 >> v2:172.21.15.137:3300/0 conn(0x7fb210071a10 0x7fb21006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f2fad8ee640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa806e790 con 0x7f2fa8071110 2024-01-19T04:55:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7fb215725640 1 -- 172.21.15.31:0/3812630097 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210072930 con 0x7fb21014b340 2024-01-19T04:55:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.487+0000 7f8c8adcd640 1 -- start start 2024-01-19T04:55:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f830915a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f830406e8d0 0x7f83040d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f830915a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f83040cee00 0x7f830415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f830915a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f830915a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83040d0040 con 0x7f830406e8d0 2024-01-19T04:55:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f830915a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830415c800 con 0x7f83040cee00 2024-01-19T04:55:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f830915a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040cd500 con 0x7f830406ecb0 2024-01-19T04:55:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f767a79e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7674161930 0x7f7674161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f767a79e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767406d110 con 0x7f76740cf880 2024-01-19T04:55:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f767a79e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767406d3f0 con 0x7f767406c5a0 2024-01-19T04:55:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f767a79e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7674070790 con 0x7f7674161930 2024-01-19T04:55:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8adcd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c0e1270 0x7f8c7c0534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8adcd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8c7c0e4cf0 0x7f8c7c0539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8adcd640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7c054fa0 con 0x7f8c7c054b10 2024-01-19T04:55:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8adcd640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7c055c20 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8adcd640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c055280 con 0x7f8c7c0e1270 2024-01-19T04:55:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f5329844640 1 Processor -- start 2024-01-19T04:55:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f5329844640 1 -- start start 2024-01-19T04:55:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f7673fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7674161930 0x7f7674161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8302d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f830406e8d0 0x7f83040d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8302d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f830406e8d0 0x7f83040d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36110/0 (socket says 172.21.15.31:36110) 2024-01-19T04:55:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8302d9d640 1 -- 172.21.15.31:0/1524496542 learned_addr learned my addr 172.21.15.31:0/1524496542 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f830359e640 1 --2- 172.21.15.31:0/1524496542 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f2fa6ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2fa80cea50 0x7f2fa806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56874/0 (socket says 172.21.15.31:56874) 2024-01-19T04:55:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f2fa77fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2fa8071110 0x7f2fa806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f2fa67fc640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f2fa806bc90 0x7f2fa8070a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:46366/0 (socket says 172.21.15.31:46366) 2024-01-19T04:55:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f2fa6ffd640 1 -- 172.21.15.31:0/4136992050 learned_addr learned my addr 172.21.15.31:0/4136992050 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8a5cc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8c7c0e4cf0 0x7f8c7c0539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8a5cc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8c7c0e4cf0 0x7f8c7c0539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34826/0 (socket says 172.21.15.31:34826) 2024-01-19T04:55:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c8a5cc640 1 -- 172.21.15.31:0/3919732603 learned_addr learned my addr 172.21.15.31:0/3919732603 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.488+0000 7f8c89dcb640 1 --2- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c0e1270 0x7f8c7c0534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f7673fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7674161930 0x7f7674161d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34816/0 (socket says 172.21.15.31:34816) 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f7673fff640 1 -- 172.21.15.31:0/93019741 learned_addr learned my addr 172.21.15.31:0/93019741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7efdbc741640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efdb4161930 0x7efdb4161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7efdbc741640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efdb40cf880 0x7efdb406ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7efdb9cdc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efdb406c5a0 0x7efdb40cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56914/0 (socket says 172.21.15.31:56914) 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7efdb9cdc640 1 -- 172.21.15.31:0/3350076011 learned_addr learned my addr 172.21.15.31:0/3350076011 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb406d110 con 0x7efdb406c5a0 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb406d3f0 con 0x7efdb4161930 2024-01-19T04:55:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb4070790 con 0x7efdb40cf880 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b1599e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2b0803e830 0x7f2b08053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b1499c640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f2b080e30b0 0x7f2b08057d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:46368/0 (socket says 172.21.15.31:46368) 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b0ffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2b0803e830 0x7f2b08053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b0ffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2b0803e830 0x7f2b08053be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34838/0 (socket says 172.21.15.31:34838) 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b0ffff640 1 -- 172.21.15.31:0/178302487 learned_addr learned my addr 172.21.15.31:0/178302487 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b08058560 con 0x7f2b08054120 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b08058840 con 0x7f2b080e30b0 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b08006190 con 0x7f2b0803e830 2024-01-19T04:55:28.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f830359e640 1 -- 172.21.15.31:0/1524496542 >> v2:172.21.15.137:3300/0 conn(0x7f83040cee00 msgr2=0x7f830415c2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f830359e640 1 --2- 172.21.15.31:0/1524496542 >> v2:172.21.15.137:3300/0 conn(0x7f83040cee00 0x7f830415c2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f830359e640 1 -- 172.21.15.31:0/1524496542 >> v2:172.21.15.31:3300/0 conn(0x7f830406e8d0 msgr2=0x7f83040d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f830359e640 1 --2- 172.21.15.31:0/1524496542 >> v2:172.21.15.31:3300/0 conn(0x7f830406e8d0 0x7f83040d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f830359e640 1 -- 172.21.15.31:0/1524496542 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040cb800 con 0x7f830406ecb0 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f5329844640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f531c0e1270 0x7f531c0534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f5328842640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f531c0e1270 0x7f531c0534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f5328842640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f531c0e1270 0x7f531c0534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36130/0 (socket says 172.21.15.31:36130) 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f5328842640 1 -- 172.21.15.31:0/1714095742 learned_addr learned my addr 172.21.15.31:0/1714095742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c054fa0 con 0x7f531c054b10 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c055280 con 0x7f531c0e4cf0 2024-01-19T04:55:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c055c20 con 0x7f531c0e1270 2024-01-19T04:55:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f8c8a5cc640 1 -- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c0e1270 msgr2=0x7f8c7c0534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f8c8a5cc640 1 --2- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c0e1270 0x7f8c7c0534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f8c8a5cc640 1 -- 172.21.15.31:0/3919732603 >> v1:172.21.15.31:6789/0 conn(0x7f8c7c054b10 legacy=0x7f8c7c0e1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f8c8a5cc640 1 -- 172.21.15.31:0/3919732603 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c055560 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7fb215725640 1 --2- 172.21.15.31:0/3812630097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb21014b340 0x7fb2100714d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb21006b2e0 tx=0x7fb20c02ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e550c1b69c94bde server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7fb207fff640 1 -- 172.21.15.31:0/3812630097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c02d640 con 0x7fb21014b340 2024-01-19T04:55:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7fb207fff640 1 -- 172.21.15.31:0/3812630097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c02d7a0 con 0x7fb21014b340 2024-01-19T04:55:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f7673fff640 1 -- 172.21.15.31:0/93019741 >> v1:172.21.15.137:6789/0 conn(0x7f767406c5a0 legacy=0x7f76740cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f7673fff640 1 -- 172.21.15.31:0/93019741 >> v1:172.21.15.31:6789/0 conn(0x7f76740cf880 legacy=0x7f7674070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7efdba4dd640 1 --2- 172.21.15.31:0/3350076011 >> v2:172.21.15.31:3300/0 conn(0x7efdb4161930 0x7efdb4161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7efdbacde640 1 --2- 172.21.15.31:0/3350076011 >> v2:172.21.15.137:3300/0 conn(0x7efdb40cf880 0x7efdb406ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f7673fff640 1 -- 172.21.15.31:0/93019741 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674070a70 con 0x7f7674161930 2024-01-19T04:55:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2fa5ffb640 1 -- 172.21.15.31:0/4136992050 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421054822 0 0) 0x7f2fa806d8c0 con 0x7f2fa806bc90 2024-01-19T04:55:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2fa5ffb640 1 -- 172.21.15.31:0/4136992050 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f90003520 con 0x7f2fa806bc90 2024-01-19T04:55:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2fa77fe640 1 -- 172.21.15.31:0/4136992050 >> v1:172.21.15.137:6789/0 conn(0x7f2fa806bc90 legacy=0x7f2fa8070a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2fa77fe640 1 -- 172.21.15.31:0/4136992050 >> v1:172.21.15.31:6789/0 conn(0x7f2fa80cea50 legacy=0x7f2fa806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.489+0000 7f2fa77fe640 1 -- 172.21.15.31:0/4136992050 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa806ab90 con 0x7f2fa8071110 2024-01-19T04:55:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f8c8a5cc640 1 --2- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c0539f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8c840621d0 tx=0x7f8c840643a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fabe30a3cde7316 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7fb217989640 1 -- 172.21.15.31:0/3812630097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb21014b340 msgr2=0x7fb2100714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f830359e640 1 --2- 172.21.15.31:0/1524496542 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040cd9e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83040c4c10 tx=0x7f82f802fbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa5e0d0d601c2a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f8c88dc9640 1 -- 172.21.15.31:0/3919732603 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840a6d80 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7fb217989640 1 --2- 172.21.15.31:0/3812630097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb21014b340 0x7fb2100714d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb21006b2e0 tx=0x7fb20c02ba90 comp rx=0 tx=0).stop 2024-01-19T04:55:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f8c88dc9640 1 -- 172.21.15.31:0/3919732603 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c840ae440 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f8c88dc9640 1 -- 172.21.15.31:0/3919732603 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840a6d80 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7efdb94db640 1 -- 172.21.15.31:0/3350076011 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287670178 0 0) 0x7efdb406d110 con 0x7efdb406c5a0 2024-01-19T04:55:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7efdb94db640 1 -- 172.21.15.31:0/3350076011 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efda0003520 con 0x7efdb406c5a0 2024-01-19T04:55:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7fb217989640 1 -- 172.21.15.31:0/3812630097 shutdown_connections 2024-01-19T04:55:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7fb217989640 1 --2- 172.21.15.31:0/3812630097 >> v2:172.21.15.137:3300/0 conn(0x7fb210071a10 0x7fb21006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7fb217989640 1 --2- 172.21.15.31:0/3812630097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21006aaf0 0x7fb210165460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7fb217989640 1 --2- 172.21.15.31:0/3812630097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb21014b340 0x7fb2100714d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7fb217989640 1 -- 172.21.15.31:0/3812630097 >> 172.21.15.31:0/3812630097 conn(0x7fb21006edc0 msgr2=0x7fb21006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f8301d9b640 1 -- 172.21.15.31:0/1524496542 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8030770 con 0x7f830406ecb0 2024-01-19T04:55:28.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f8301d9b640 1 -- 172.21.15.31:0/1524496542 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f80308d0 con 0x7f830406ecb0 2024-01-19T04:55:28.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f2fa77fe640 1 --2- 172.21.15.31:0/4136992050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa8071110 0x7f2fa806a5b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009d90 tx=0x7f2fa002bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6437ef3b97d5d46 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f2b0ffff640 1 -- 172.21.15.31:0/178302487 >> v1:172.21.15.137:6789/0 conn(0x7f2b080e30b0 legacy=0x7f2b08057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f2b0ffff640 1 -- 172.21.15.31:0/178302487 >> v1:172.21.15.31:6789/0 conn(0x7f2b08054120 legacy=0x7f2b080068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.490+0000 7f2b0ffff640 1 -- 172.21.15.31:0/178302487 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08058b20 con 0x7f2b0803e830 2024-01-19T04:55:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165858651 0 0) 0x7f531c055280 con 0x7f531c0e4cf0 2024-01-19T04:55:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5314003520 con 0x7f531c0e4cf0 2024-01-19T04:55:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7fb217989640 1 -- 172.21.15.31:0/3812630097 shutdown_connections 2024-01-19T04:55:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7fb217989640 1 -- 172.21.15.31:0/3812630097 wait complete. 2024-01-19T04:55:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f8301d9b640 1 -- 172.21.15.31:0/1524496542 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8030770 con 0x7f830406ecb0 2024-01-19T04:55:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f2fa5ffb640 1 -- 172.21.15.31:0/4136992050 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa002ccf0 con 0x7f2fa8071110 2024-01-19T04:55:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f2fa5ffb640 1 -- 172.21.15.31:0/4136992050 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa002ce50 con 0x7f2fa8071110 2024-01-19T04:55:28.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f2fa5ffb640 1 -- 172.21.15.31:0/4136992050 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0034660 con 0x7f2fa8071110 2024-01-19T04:55:28.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7efdbacde640 1 -- 172.21.15.31:0/3350076011 >> v2:172.21.15.31:3300/0 conn(0x7efdb4161930 msgr2=0x7efdb4161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7efdbacde640 1 --2- 172.21.15.31:0/3350076011 >> v2:172.21.15.31:3300/0 conn(0x7efdb4161930 0x7efdb4161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f830915a640 1 -- 172.21.15.31:0/1524496542 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 msgr2=0x7f83040cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f830915a640 1 --2- 172.21.15.31:0/1524496542 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040cd9e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83040c4c10 tx=0x7f82f802fbd0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f830915a640 1 -- 172.21.15.31:0/1524496542 shutdown_connections 2024-01-19T04:55:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f830915a640 1 --2- 172.21.15.31:0/1524496542 >> v2:172.21.15.137:3300/0 conn(0x7f83040cee00 0x7f830415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f830915a640 1 --2- 172.21.15.31:0/1524496542 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040cd9e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f830915a640 1 --2- 172.21.15.31:0/1524496542 >> v2:172.21.15.31:3300/0 conn(0x7f830406e8d0 0x7f83040d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f830915a640 1 -- 172.21.15.31:0/1524496542 >> 172.21.15.31:0/1524496542 conn(0x7f83040b2c60 msgr2=0x7f83040b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2b0ffff640 1 --2- 172.21.15.31:0/178302487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 0x7f2b08053be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009870 tx=0x7f2afc02ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e14484a12ff4c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f7673fff640 1 --2- 172.21.15.31:0/93019741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 0x7f7674161d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f766c009e50 tx=0x7f766c02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd9530c3654891b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f7672ffd640 1 -- 172.21.15.31:0/93019741 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c02c9f0 con 0x7f7674161930 2024-01-19T04:55:28.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f7672ffd640 1 -- 172.21.15.31:0/93019741 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c02cb50 con 0x7f7674161930 2024-01-19T04:55:28.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f7672ffd640 1 -- 172.21.15.31:0/93019741 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c02c9f0 con 0x7f7674161930 2024-01-19T04:55:28.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f830915a640 1 -- 172.21.15.31:0/1524496542 shutdown_connections 2024-01-19T04:55:28.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f830915a640 1 -- 172.21.15.31:0/1524496542 wait complete. 2024-01-19T04:55:28.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 <== mon.1 v1:172.21.15.137:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3229524749 0 0) 0x7f5314003520 con 0x7f531c0e4cf0 2024-01-19T04:55:28.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7efdbacde640 1 -- 172.21.15.31:0/3350076011 >> v1:172.21.15.31:6789/0 conn(0x7efdb406c5a0 legacy=0x7efdb40cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7efdbacde640 1 -- 172.21.15.31:0/3350076011 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4070a70 con 0x7efdb40cf880 2024-01-19T04:55:28.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2b0f7fe640 1 -- 172.21.15.31:0/178302487 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc02f850 con 0x7f2b0803e830 2024-01-19T04:55:28.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 >> v2:172.21.15.31:3300/0 conn(0x7f531c0e1270 msgr2=0x7f531c0534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f767a79e640 1 -- 172.21.15.31:0/93019741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 msgr2=0x7f7674161d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f767a79e640 1 --2- 172.21.15.31:0/93019741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 0x7f7674161d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f766c009e50 tx=0x7f766c02be50 comp rx=0 tx=0).stop 2024-01-19T04:55:28.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f767a79e640 1 -- 172.21.15.31:0/93019741 shutdown_connections 2024-01-19T04:55:28.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f767a79e640 1 --2- 172.21.15.31:0/93019741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 0x7f7674161d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f767a79e640 1 -- 172.21.15.31:0/93019741 >> 172.21.15.31:0/93019741 conn(0x7f767406a790 msgr2=0x7f767406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f767a79e640 1 -- 172.21.15.31:0/93019741 shutdown_connections 2024-01-19T04:55:28.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f767a79e640 1 -- 172.21.15.31:0/93019741 wait complete. 2024-01-19T04:55:28.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7efdbacde640 1 --2- 172.21.15.31:0/3350076011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb40cf880 0x7efdb406ca00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdb406de00 tx=0x7efdac02bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1d54b9ccd90ea7 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2b0f7fe640 1 -- 172.21.15.31:0/178302487 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc02f9b0 con 0x7f2b0803e830 2024-01-19T04:55:28.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2b0f7fe640 1 -- 172.21.15.31:0/178302487 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc02fd40 con 0x7f2b0803e830 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 --2- 172.21.15.31:0/1714095742 >> v2:172.21.15.31:3300/0 conn(0x7f531c0e1270 0x7f531c0534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 >> v1:172.21.15.31:6789/0 conn(0x7f531c054b10 legacy=0x7f531c0e1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 --> v1:172.21.15.137:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c055560 con 0x7f531c0e4cf0 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 <== mon.1 v1:172.21.15.137:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f532404f1c0 con 0x7f531c0e4cf0 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.491+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 >> v1:172.21.15.137:6789/0 conn(0x7f531c0e4cf0 legacy=0x7f531c0e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f53237fe640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5314003970 0x7f5314003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f53237fe640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5314004310 0x7f53140047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f53237fe640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5314004d20 0x7f53140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c055c20 con 0x7f5314004310 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5314014720 con 0x7f5314004d20 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53140148e0 con 0x7f5314003970 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f5328842640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5314003970 0x7f5314003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f5329043640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5314004d20 0x7f53140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f5328842640 1 -- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5314004d20 msgr2=0x7f53140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f5328842640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5314004d20 0x7f53140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f5328842640 1 -- 172.21.15.31:0/1714095742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5314004310 msgr2=0x7f53140047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f5328842640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5314004310 0x7f53140047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f5328842640 1 -- 172.21.15.31:0/1714095742 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314003520 con 0x7f5314003970 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2fad8ee640 1 -- 172.21.15.31:0/4136992050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa8071110 msgr2=0x7f2fa806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2fad8ee640 1 --2- 172.21.15.31:0/4136992050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa8071110 0x7f2fa806a5b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009d90 tx=0x7f2fa002bf40 comp rx=0 tx=0).stop 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2fad8ee640 1 -- 172.21.15.31:0/4136992050 shutdown_connections 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2fad8ee640 1 --2- 172.21.15.31:0/4136992050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa8071110 0x7f2fa806a5b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2fad8ee640 1 -- 172.21.15.31:0/4136992050 >> 172.21.15.31:0/4136992050 conn(0x7f2fa80b1a20 msgr2=0x7f2fa80cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f2fad8ee640 1 -- 172.21.15.31:0/4136992050 shutdown_connections 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f2fad8ee640 1 -- 172.21.15.31:0/4136992050 wait complete. 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f8c8adcd640 1 -- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e4cf0 msgr2=0x7f8c7c0539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.492+0000 7f8c8adcd640 1 --2- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c0539f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8c840621d0 tx=0x7f8c840643a0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f8c8adcd640 1 -- 172.21.15.31:0/3919732603 shutdown_connections 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f8c8adcd640 1 --2- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c0539f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f8c8adcd640 1 --2- 172.21.15.31:0/3919732603 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c0e1270 0x7f8c7c0534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f8c8adcd640 1 -- 172.21.15.31:0/3919732603 >> 172.21.15.31:0/3919732603 conn(0x7f8c7c020000 msgr2=0x7f8c7c056d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7fb217989640 1 Processor -- start 2024-01-19T04:55:28.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7fb217989640 1 -- start start 2024-01-19T04:55:28.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7efdb94db640 1 -- 172.21.15.31:0/3350076011 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac02cb50 con 0x7efdb40cf880 2024-01-19T04:55:28.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7efdb94db640 1 -- 172.21.15.31:0/3350076011 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac02ccb0 con 0x7efdb40cf880 2024-01-19T04:55:28.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7efdb94db640 1 -- 172.21.15.31:0/3350076011 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac034660 con 0x7efdb40cf880 2024-01-19T04:55:28.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f8c8adcd640 1 -- 172.21.15.31:0/3919732603 shutdown_connections 2024-01-19T04:55:28.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 msgr2=0x7f2b08053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f2b1599e640 1 --2- 172.21.15.31:0/178302487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 0x7f2b08053be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009870 tx=0x7f2afc02ba30 comp rx=0 tx=0).stop 2024-01-19T04:55:28.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f5329043640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5314004d20 0x7f53140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:28.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f5329043640 1 -- 172.21.15.31:0/1714095742 reap_dead start 2024-01-19T04:55:28.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f5328842640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5314003970 0x7f5314003d90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5310009c10 tx=0x7f53140057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d238757928724819 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.493+0000 7f8c8adcd640 1 -- 172.21.15.31:0/3919732603 wait complete. 2024-01-19T04:55:28.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531003d9f0 con 0x7f5314003970 2024-01-19T04:55:28.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531003db50 con 0x7f5314003970 2024-01-19T04:55:28.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f53237fe640 1 -- 172.21.15.31:0/1714095742 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310045440 con 0x7f5314003970 2024-01-19T04:55:28.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb40cf880 msgr2=0x7efdb406ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7efdbc741640 1 --2- 172.21.15.31:0/3350076011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb40cf880 0x7efdb406ca00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdb406de00 tx=0x7efdac02bda0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 shutdown_connections 2024-01-19T04:55:28.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f2b1599e640 1 --2- 172.21.15.31:0/178302487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 0x7f2b08053be0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 >> 172.21.15.31:0/178302487 conn(0x7f2b080559d0 msgr2=0x7f2b08055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f830915a640 1 Processor -- start 2024-01-19T04:55:28.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 shutdown_connections 2024-01-19T04:55:28.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7fb217989640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21006aaf0 0x7fb2101412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f830915a640 1 -- start start 2024-01-19T04:55:28.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f2b1599e640 1 -- 172.21.15.31:0/178302487 wait complete. 2024-01-19T04:55:28.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f2fad8ee640 1 Processor -- start 2024-01-19T04:55:28.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.494+0000 7f767a79e640 1 Processor -- start 2024-01-19T04:55:28.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f2fad8ee640 1 -- start start 2024-01-19T04:55:28.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f767a79e640 1 -- start start 2024-01-19T04:55:28.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 shutdown_connections 2024-01-19T04:55:28.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7efdbc741640 1 --2- 172.21.15.31:0/3350076011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb40cf880 0x7efdb406ca00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7efdbc741640 1 --2- 172.21.15.31:0/3350076011 >> v2:172.21.15.31:3300/0 conn(0x7efdb4161930 0x7efdb4161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 >> 172.21.15.31:0/3350076011 conn(0x7efdb406a790 msgr2=0x7efdb406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f8c8adcd640 1 Processor -- start 2024-01-19T04:55:28.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f8c8adcd640 1 -- start start 2024-01-19T04:55:28.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 shutdown_connections 2024-01-19T04:55:28.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f830915a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f830406e8d0 0x7f830406d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f830915a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f830915a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83040cee00 0x7f83041f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5314003970 msgr2=0x7f5314003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f5329844640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5314003970 0x7f5314003d90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5310009c10 tx=0x7f53140057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f2fad8ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa806bc90 0x7f2fa8163380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f830915a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83040ca9a0 con 0x7f83040cee00 2024-01-19T04:55:28.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f830915a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83040c9d20 con 0x7f830406e8d0 2024-01-19T04:55:28.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f830915a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040ca000 con 0x7f830406ecb0 2024-01-19T04:55:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f2fad8ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa8071110 0x7f2fa8155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.495+0000 7f830359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83040cee00 0x7f83041f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f8302d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f830406e8d0 0x7f830406d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f830359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83040cee00 0x7f83041f4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36136/0 (socket says 172.21.15.31:36136) 2024-01-19T04:55:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f830359e640 1 -- 172.21.15.31:0/1007264697 learned_addr learned my addr 172.21.15.31:0/1007264697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb217989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210071a10 0x7fb210149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7efdbc741640 1 -- 172.21.15.31:0/3350076011 wait complete. 2024-01-19T04:55:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fad8ee640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fa80cea50 0x7f2fa8142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb217989640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21014b340 0x7fb21014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fad8ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8155bb0 con 0x7f2fa8071110 2024-01-19T04:55:28.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fad8ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81584c0 con 0x7f2fa806bc90 2024-01-19T04:55:28.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fad8ee640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81523e0 con 0x7f2fa80cea50 2024-01-19T04:55:28.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa806bc90 0x7f2fa8163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa806bc90 0x7f2fa8163380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34878/0 (socket says 172.21.15.31:34878) 2024-01-19T04:55:28.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 -- 172.21.15.31:0/3370054163 learned_addr learned my addr 172.21.15.31:0/3370054163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb217989640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210145cf0 con 0x7fb210071a10 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb217989640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210145140 con 0x7fb21006aaf0 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb214f24640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210071a10 0x7fb210149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb217989640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101fb7b0 con 0x7fb21014b340 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb214f24640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210071a10 0x7fb210149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36154/0 (socket says 172.21.15.31:36154) 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f767a79e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f767406c5a0 0x7f7674153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215f26640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21014b340 0x7fb21014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb214f24640 1 -- 172.21.15.31:0/3885492014 learned_addr learned my addr 172.21.15.31:0/3885492014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f767a79e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76740cf880 0x7f76741532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa77fe640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fa80cea50 0x7f2fa8142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f8302d9d640 1 -- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 msgr2=0x7f83040c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f830259c640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f8302d9d640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f8302d9d640 1 -- 172.21.15.31:0/1007264697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83040cee00 msgr2=0x7f83041f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f8302d9d640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83040cee00 0x7f83041f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f8302d9d640 1 -- 172.21.15.31:0/1007264697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f800d8a0 con 0x7f830406e8d0 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f830259c640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f830406ecb0 0x7f83040c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f767a79e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 0x7f76741e9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f767a79e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76741585d0 con 0x7f76740cf880 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f767a79e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76741eb560 con 0x7f7674161930 2024-01-19T04:55:28.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f767a79e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7674158250 con 0x7f767406c5a0 2024-01-19T04:55:28.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa67fc640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa8071110 0x7f2fa8155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 -- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fa80cea50 msgr2=0x7f2fa8142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fa80cea50 0x7f2fa8142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 -- 172.21.15.31:0/3370054163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa8071110 msgr2=0x7f2fa8155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa8071110 0x7f2fa8155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f2fa6ffd640 1 -- 172.21.15.31:0/3370054163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0009a70 con 0x7f2fa806bc90 2024-01-19T04:55:28.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215725640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21006aaf0 0x7fb2101412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215f26640 1 -- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21006aaf0 msgr2=0x7fb2101412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215f26640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21006aaf0 0x7fb2101412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215f26640 1 -- 172.21.15.31:0/3885492014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210071a10 msgr2=0x7fb210149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215f26640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210071a10 0x7fb210149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215f26640 1 -- 172.21.15.31:0/3885492014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c009400 con 0x7fb21014b340 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7fb215725640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21006aaf0 0x7fb2101412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7fb214f24640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210071a10 0x7fb210149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f7678d3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 0x7f76741e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f76737fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76740cf880 0x7f76741532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f7678d3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 0x7f76741e9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34902/0 (socket says 172.21.15.31:34902) 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f7678d3b640 1 -- 172.21.15.31:0/1841666727 learned_addr learned my addr 172.21.15.31:0/1841666727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f7673fff640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f767406c5a0 0x7f7674153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2b1599e640 1 Processor -- start 2024-01-19T04:55:28.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2b1599e640 1 -- start start 2024-01-19T04:55:28.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2fa6ffd640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa806bc90 0x7f2fa8163380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c480 tx=0x7f2f9800c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.496+0000 7f8302d9d640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f830406e8d0 0x7f830406d010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e870 tx=0x7f82f400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4009e40 con 0x7f830406e8d0 2024-01-19T04:55:28.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f400c940 con 0x7f830406e8d0 2024-01-19T04:55:28.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f400ccf0 con 0x7f830406e8d0 2024-01-19T04:55:28.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7fb215f26640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21014b340 0x7fb21014a120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb20800eab0 tx=0x7fb20800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f76737fe640 1 -- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f767406c5a0 msgr2=0x7f7674153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f76737fe640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f767406c5a0 0x7f7674153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f76737fe640 1 -- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 msgr2=0x7f76741e9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f76737fe640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7674161930 0x7f76741e9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f76737fe640 1 -- 172.21.15.31:0/1841666727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c009b30 con 0x7f76740cf880 2024-01-19T04:55:28.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f830915a640 1 -- 172.21.15.31:0/1007264697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83040ca2e0 con 0x7f830406e8d0 2024-01-19T04:55:28.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f830915a640 1 -- 172.21.15.31:0/1007264697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830406d550 con 0x7f830406e8d0 2024-01-19T04:55:28.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f8c8adcd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e1270 0x7f8c7c180990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 shutdown_connections 2024-01-19T04:55:28.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f5329844640 1 --2- 172.21.15.31:0/1714095742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5314003970 0x7f5314003d90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 >> 172.21.15.31:0/1714095742 conn(0x7f531c020000 msgr2=0x7f531c056d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208010040 con 0x7fb21014b340 2024-01-19T04:55:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98005ce0 con 0x7f2fa806bc90 2024-01-19T04:55:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98005e40 con 0x7f2fa806bc90 2024-01-19T04:55:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980072f0 con 0x7f2fa806bc90 2024-01-19T04:55:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa81526c0 con 0x7f2fa806bc90 2024-01-19T04:55:28.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb208003b90 con 0x7fb21014b340 2024-01-19T04:55:28.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20800b7f0 con 0x7fb21014b340 2024-01-19T04:55:28.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.497+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa8151c30 con 0x7f2fa806bc90 2024-01-19T04:55:28.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7efdbc741640 1 Processor -- start 2024-01-19T04:55:28.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb210141800 con 0x7fb21014b340 2024-01-19T04:55:28.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c8adcd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c04f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7efdbc741640 1 -- start start 2024-01-19T04:55:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c8adcd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c03c310 0x7f8c7c0de460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb210145630 con 0x7fb21014b340 2024-01-19T04:55:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c8adcd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c181930 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c895ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c04f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c895ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c04f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36182/0 (socket says 172.21.15.31:36182) 2024-01-19T04:55:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f76737fe640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76740cf880 0x7f76741532e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f766800c440 tx=0x7f766800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c895ca640 1 -- 172.21.15.31:0/2402935440 learned_addr learned my addr 172.21.15.31:0/2402935440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 shutdown_connections 2024-01-19T04:55:28.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c8a5cc640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c03c310 0x7f8c7c0de460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0e3400 con 0x7f8c7c0e1270 2024-01-19T04:55:28.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c04f6b0 con 0x7f8c7c03c310 2024-01-19T04:55:28.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f5329844640 1 -- 172.21.15.31:0/1714095742 wait complete. 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.498+0000 7f2b1599e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 0x7f2b080daf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1599e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b08054120 0x7f2b0817c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1599e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b080e30b0 0x7f2b080debf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1599e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08051c40 con 0x7f2b080e30b0 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1599e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0817d260 con 0x7f2b0803e830 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1599e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0817c9c0 con 0x7f2b08054120 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f8c89dcb640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e1270 0x7f8c7c180990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f830915a640 1 -- 172.21.15.31:0/1007264697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83040d0e80 con 0x7f830406e8d0 2024-01-19T04:55:28.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fa8165130 con 0x7f2fa806bc90 2024-01-19T04:55:28.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1499c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 0x7f2b080daf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7efdbc741640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efdb406c5a0 0x7efdb40aa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1519d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b080e30b0 0x7f2b080debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1519d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b080e30b0 0x7f2b080debf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36184/0 (socket says 172.21.15.31:36184) 2024-01-19T04:55:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b1519d640 1 -- 172.21.15.31:0/3444947729 learned_addr learned my addr 172.21.15.31:0/3444947729 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f2b0ffff640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b08054120 0x7f2b0817c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7efdbc741640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdb40cf880 0x7efdb40aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f82f4013470 con 0x7f830406e8d0 2024-01-19T04:55:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f2b1519d640 1 -- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b08054120 msgr2=0x7f2b0817c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f2b1519d640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b08054120 0x7f2b0817c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f2b1519d640 1 -- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 msgr2=0x7f2b080daf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f2b1519d640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0803e830 0x7f2b080daf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f2b1519d640 1 -- 172.21.15.31:0/3444947729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0094d0 con 0x7f2b080e30b0 2024-01-19T04:55:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f8c895ca640 1 -- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c03c310 msgr2=0x7f8c7c0de460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdbc741640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb4161930 0x7efdb40acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f8c895ca640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c03c310 0x7f8c7c0de460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f8c895ca640 1 -- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e1270 msgr2=0x7f8c7c180990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f8c895ca640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c7c0e1270 0x7f8c7c180990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb20800b950 con 0x7fb21014b340 2024-01-19T04:55:28.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdbc741640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40a9e10 con 0x7efdb40cf880 2024-01-19T04:55:28.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f8c895ca640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8404eee0 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f8c895ca640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c04f170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c780 tx=0x7f8c8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f8c8a5cc640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c7c03c310 0x7f8c7c0de460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:28.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdbc741640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40a9500 con 0x7efdb4161930 2024-01-19T04:55:28.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdbc741640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb41f7bc0 con 0x7efdb406c5a0 2024-01-19T04:55:28.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.499+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680035f0 con 0x7f76740cf880 2024-01-19T04:55:28.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668003770 con 0x7f76740cf880 2024-01-19T04:55:28.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668003b20 con 0x7f76740cf880 2024-01-19T04:55:28.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f2b1519d640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b080e30b0 0x7f2b080debf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b10073470 tx=0x7f2b1006b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f767a79e640 1 -- 172.21.15.31:0/1841666727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76741ec050 con 0x7f76740cf880 2024-01-19T04:55:28.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdba4dd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efdb406c5a0 0x7efdb40aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f82eb7fe640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f82f006afa0 0x7f82f006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000a840 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f767a79e640 1 -- 172.21.15.31:0/1841666727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76741eba90 con 0x7f76740cf880 2024-01-19T04:55:28.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdbacde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb4161930 0x7efdb40acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7674162250 con 0x7f76740cf880 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdbacde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb4161930 0x7efdb40acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34938/0 (socket says 172.21.15.31:34938) 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdbacde640 1 -- 172.21.15.31:0/816622369 learned_addr learned my addr 172.21.15.31:0/816622369 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7efdb9cdc640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdb40cf880 0x7efdb40aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82f40896d0 con 0x7f830406e8d0 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f830259c640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f82f006afa0 0x7f82f006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f82eb7fe640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f82f0073140 0x7f82f0075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 --> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82f0075c10 con 0x7f82f0073140 2024-01-19T04:55:28.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f40543b0 con 0x7f830406e8d0 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f5329844640 1 Processor -- start 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f5329844640 1 -- start start 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f5329844640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5314003970 0x7f531c1790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329844640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f531c0df5b0 0x7f531c04d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329844640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f531c03bd10 0x7f531c0e5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329844640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c179630 con 0x7f5314003970 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329844640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c178650 con 0x7f531c0df5b0 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329844640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c03c680 con 0x7f531c03bd10 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2f98007510 con 0x7f2fa806bc90 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f2f8ffff640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2f9406afa0 0x7f2f9406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f98088df0 con 0x7f2fa806bc90 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2f8ffff640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f2f940731c0 0x7f2f940755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 --> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f94075c90 con 0x7f2f940731c0 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f980891a0 con 0x7f2fa806bc90 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c181090 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.500+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c181650 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8000a9a0 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000ad10 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c6c001030 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb210072650 con 0x7fb21014b340 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7fb205ffb640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1f406afa0 0x7fb1f406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb208089550 con 0x7fb21014b340 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7fb205ffb640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7fb1f4073140 0x7fb1f4075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 --> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1f4075c10 con 0x7fb1f4073140 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb208054490 con 0x7fb21014b340 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.501+0000 7fb214f24640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7fb1f4073140 0x7fb1f4075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2fa67fc640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2f9406afa0 0x7f2f9406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5323fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f531c0df5b0 0x7f531c04d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f830259c640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f82f006afa0 0x7f82f006d460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f830415cfa0 tx=0x7f82fc0073a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2fa77fe640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f2f940731c0 0x7f2f940755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b100723d0 con 0x7f2b080e30b0 2024-01-19T04:55:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b10072530 con 0x7f2b080e30b0 2024-01-19T04:55:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10072720 con 0x7f2b080e30b0 2024-01-19T04:55:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5323fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f531c0df5b0 0x7f531c04d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34946/0 (socket says 172.21.15.31:34946) 2024-01-19T04:55:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f830359e640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f82f0073140 0x7f82f0075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7fb214f24640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7fb1f4073140 0x7fb1f4075560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0817cca0 con 0x7f2b080e30b0 2024-01-19T04:55:28.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5323fff640 1 -- 172.21.15.31:0/3974973614 learned_addr learned my addr 172.21.15.31:0/3974973614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329043640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f531c03bd10 0x7f531c0e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b080d3500 con 0x7f2b080e30b0 2024-01-19T04:55:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329043640 1 -- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f531c0df5b0 msgr2=0x7f531c04d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7efdbacde640 1 -- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efdb406c5a0 msgr2=0x7efdb40aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7efdbacde640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efdb406c5a0 0x7efdb40aa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7efdbacde640 1 -- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdb40cf880 msgr2=0x7efdb40aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7efdbacde640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdb40cf880 0x7efdb40aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7efdbacde640 1 -- 172.21.15.31:0/816622369 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac009a70 con 0x7efdb4161930 2024-01-19T04:55:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7efdb9cdc640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdb40cf880 0x7efdb40aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:28.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7efdbacde640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb4161930 0x7efdb40acac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdac02c3b0 tx=0x7efdac004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329043640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f531c0df5b0 0x7f531c04d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329043640 1 -- 172.21.15.31:0/3974973614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5314003970 msgr2=0x7f531c1790f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329043640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5314003970 0x7f531c1790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.502+0000 7f5329043640 1 -- 172.21.15.31:0/3974973614 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c03c960 con 0x7f531c03bd10 2024-01-19T04:55:28.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7f830359e640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f82f0073140 0x7f82f0075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac03e610 con 0x7efdb4161930 2024-01-19T04:55:28.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7fb215725640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1f406afa0 0x7fb1f406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac035e90 con 0x7efdb4161930 2024-01-19T04:55:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb40a9750 con 0x7efdb4161930 2024-01-19T04:55:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7f2fa67fc640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2f9406afa0 0x7f2f9406d460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2fa81648f0 tx=0x7f2f9c007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb40aa770 con 0x7efdb4161930 2024-01-19T04:55:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7f5329043640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f531c03bd10 0x7f531c0e5160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5324069420 tx=0x7f532406b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.503+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== osd.7 v2:172.21.15.137:6800/1281133662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb1f4075c10 con 0x7fb1f4073140 2024-01-19T04:55:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac03e9d0 con 0x7efdb4161930 2024-01-19T04:55:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7668003c80 con 0x7f76740cf880 2024-01-19T04:55:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53240723e0 con 0x7f531c03bd10 2024-01-19T04:55:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7f7670ff9640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f766006af20 0x7f766006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76680889c0 con 0x7f76740cf880 2024-01-19T04:55:28.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== osd.3 v2:172.21.15.31:6809/3034750361 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f82f0075c10 con 0x7f82f0073140 2024-01-19T04:55:28.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f7670ff9640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f7660073140 0x7f7660075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 --> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7660075c10 con 0x7f7660073140 2024-01-19T04:55:28.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f766808d2c0 con 0x7f76740cf880 2024-01-19T04:55:28.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f7678d3b640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f7660073140 0x7f7660075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f2fa77fe640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f2f940731c0 0x7f2f940755e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f7673fff640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f766006af20 0x7f766006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8c80012420 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b0803bfb0 con 0x7f2b080e30b0 2024-01-19T04:55:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c1788f0 con 0x7f531c03bd10 2024-01-19T04:55:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.504+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c04d800 con 0x7f531c03bd10 2024-01-19T04:55:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5324072540 con 0x7f531c03bd10 2024-01-19T04:55:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53240728f0 con 0x7f531c03bd10 2024-01-19T04:55:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f531c0534b0 con 0x7f531c03bd10 2024-01-19T04:55:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2b10072880 con 0x7f2b080e30b0 2024-01-19T04:55:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efdb40696e0 con 0x7efdb4161930 2024-01-19T04:55:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.505+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efdac03b050 con 0x7efdb4161930 2024-01-19T04:55:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== osd.5 v2:172.21.15.137:6809/4102773142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2f94075c90 con 0x7f2f940731c0 2024-01-19T04:55:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7efdaaffd640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd9806afa0 0x7efd9806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7f2b0d7fa640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2af406afa0 0x7f2af406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7efdba4dd640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd9806afa0 0x7efd9806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7f8c7a7fc640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c6806afa0 0x7f8c6806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b101033d0 con 0x7f2b080e30b0 2024-01-19T04:55:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7f2b0d7fa640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f2af4073140 0x7f2af4075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdac0abdb0 con 0x7efdb4161930 2024-01-19T04:55:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 --> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2af4075c10 con 0x7f2af4073140 2024-01-19T04:55:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7f2b1499c640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2af406afa0 0x7f2af406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7efdba4dd640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd9806afa0 0x7efd9806d460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009aa0 tx=0x7efdb0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7f8c89dcb640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c6806afa0 0x7f8c6806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7efdaaffd640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7efd980731c0 0x7efd980755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 --> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd98075c90 con 0x7efd980731c0 2024-01-19T04:55:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.506+0000 7f7678d3b640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f7660073140 0x7f7660075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7f7673fff640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f766006af20 0x7f766006d3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f766c035000 tx=0x7f766c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== osd.0 v2:172.21.15.31:6808/2694559820 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7660075c10 con 0x7f7660073140 2024-01-19T04:55:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdac0ae3d0 con 0x7efdb4161930 2024-01-19T04:55:28.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b100ce310 con 0x7f2b080e30b0 2024-01-19T04:55:28.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7fb215725640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1f406afa0 0x7fb1f406d460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb20c036000 tx=0x7fb20c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7efdb9cdc640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7efd980731c0 0x7efd980755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.507+0000 7f2b0ffff640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f2af4073140 0x7f2af4075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7efdb9cdc640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7efd980731c0 0x7efd980755e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7f2b1499c640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2af406afa0 0x7f2af406d460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b04007fd0 tx=0x7f2b04007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7f2b0ffff640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f2af4073140 0x7f2af4075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5324072a50 con 0x7f531c03bd10 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f53217fa640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f530806af50 0x7f530806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c80089c80 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7f8c7a7fc640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f8c68073050 0x7f8c68075470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c68075b20 con 0x7f8c68073050 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.508+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c8008a030 con 0x7f8c7c0e4cf0 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f8c8a5cc640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f8c68073050 0x7f8c68075470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5324103940 con 0x7f531c03bd10 2024-01-19T04:55:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== osd.2 v2:172.21.15.31:6824/4257663237 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2af4075c10 con 0x7f2af4073140 2024-01-19T04:55:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f53217fa640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f5308073170 0x7f5308075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== osd.1 v2:172.21.15.31:6800/825319266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd98075c90 con 0x7efd980731c0 2024-01-19T04:55:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 --> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5308075c40 con 0x7f5308073170 2024-01-19T04:55:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f8c8a5cc640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f8c68073050 0x7f8c68075470 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5324103d60 con 0x7f531c03bd10 2024-01-19T04:55:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.509+0000 7f5328842640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f530806af50 0x7f530806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.510+0000 7f5328842640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f530806af50 0x7f530806d410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5310047000 tx=0x7f531003f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.510+0000 7f8c89dcb640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c6806afa0 0x7f8c6806d460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009aa0 tx=0x7f8c74009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.510+0000 7f5323fff640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f5308073170 0x7f5308075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.510+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== osd.6 v2:172.21.15.137:6808/4246993982 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8c68075b20 con 0x7f8c68073050 2024-01-19T04:55:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.510+0000 7f5323fff640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f5308073170 0x7f5308075590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.511+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== osd.4 v2:172.21.15.137:6814/7950289 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5308075c40 con 0x7f5308073170 2024-01-19T04:55:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.535+0000 7f830915a640 1 -- 172.21.15.31:0/1007264697 --> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83041591e0 con 0x7f82f0073140 2024-01-19T04:55:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.536+0000 7f82eb7fe640 1 -- 172.21.15.31:0/1007264697 <== osd.3 v2:172.21.15.31:6809/3034750361 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f82f800c040 con 0x7f82f0073140 2024-01-19T04:55:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.536+0000 7f82e97fa640 1 -- 172.21.15.31:0/1007264697 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f82f0073140 msgr2=0x7f82f0075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.536+0000 7f82e97fa640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f82f0073140 0x7f82f0075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.536+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 --> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb210072370 con 0x7fb1f4073140 2024-01-19T04:55:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.536+0000 7fb205ffb640 1 -- 172.21.15.31:0/3885492014 <== osd.7 v2:172.21.15.137:6800/1281133662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb2000095e0 con 0x7fb1f4073140 2024-01-19T04:55:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 -- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f82f006afa0 msgr2=0x7f82f006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f82f006afa0 0x7f82f006d460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f830415cfa0 tx=0x7f82fc0073a0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 -- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f830406e8d0 msgr2=0x7f830406d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 --2- 172.21.15.31:0/1007264697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f830406e8d0 0x7f830406d010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e870 tx=0x7f82f400ed30 comp rx=0 tx=0).stop 2024-01-19T04:55:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f830359e640 1 -- 172.21.15.31:0/1007264697 reap_dead start 2024-01-19T04:55:28.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 -- 172.21.15.31:0/1007264697 shutdown_connections 2024-01-19T04:55:28.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 -- 172.21.15.31:0/1007264697 >> 172.21.15.31:0/1007264697 conn(0x7f83040b2c60 msgr2=0x7f83040b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 -- 172.21.15.31:0/1007264697 shutdown_connections 2024-01-19T04:55:28.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7f82e97fa640 1 -- 172.21.15.31:0/1007264697 wait complete. 2024-01-19T04:55:28.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7fb1f4073140 msgr2=0x7fb1f4075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.537+0000 7fb217989640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7fb1f4073140 0x7fb1f4075560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1f406afa0 msgr2=0x7fb1f406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1f406afa0 0x7fb1f406d460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb20c036000 tx=0x7fb20c008030 comp rx=0 tx=0).stop 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21014b340 msgr2=0x7fb21014a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 --2- 172.21.15.31:0/3885492014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21014b340 0x7fb21014a120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb20800eab0 tx=0x7fb20800ef70 comp rx=0 tx=0).stop 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb215f26640 1 -- 172.21.15.31:0/3885492014 reap_dead start 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 shutdown_connections 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 >> 172.21.15.31:0/3885492014 conn(0x7fb21006edc0 msgr2=0x7fb210160040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 shutdown_connections 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.538+0000 7fb217989640 1 -- 172.21.15.31:0/3885492014 wait complete. 2024-01-19T04:55:28.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.541+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 --> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fa81446e0 con 0x7f2f940731c0 2024-01-19T04:55:28.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.541+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 --> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b08054590 con 0x7f2af4073140 2024-01-19T04:55:28.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.541+0000 7f2f8ffff640 1 -- 172.21.15.31:0/3370054163 <== osd.5 v2:172.21.15.137:6809/4102773142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2fa81446e0 con 0x7f2f940731c0 2024-01-19T04:55:28.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.541+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 --> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efdb40a9f50 con 0x7efd980731c0 2024-01-19T04:55:28.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.541+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 --> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c6c002d30 con 0x7f8c68073050 2024-01-19T04:55:28.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.541+0000 7f2b0d7fa640 1 -- 172.21.15.31:0/3444947729 <== osd.2 v2:172.21.15.31:6824/4257663237 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2afc02fb20 con 0x7f2af4073140 2024-01-19T04:55:28.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.541+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 --> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f76740696e0 con 0x7f7660073140 2024-01-19T04:55:28.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7efdaaffd640 1 -- 172.21.15.31:0/816622369 <== osd.1 v2:172.21.15.31:6800/825319266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efdb40a9f50 con 0x7efd980731c0 2024-01-19T04:55:28.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f8c7a7fc640 1 -- 172.21.15.31:0/2402935440 <== osd.6 v2:172.21.15.137:6808/4246993982 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8c6c002d30 con 0x7f8c68073050 2024-01-19T04:55:28.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f2f940731c0 msgr2=0x7f2f940755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f2f940731c0 0x7f2f940755e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f7670ff9640 1 -- 172.21.15.31:0/1841666727 <== osd.0 v2:172.21.15.31:6808/2694559820 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f76740696e0 con 0x7f7660073140 2024-01-19T04:55:28.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2f9406afa0 msgr2=0x7f2f9406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2f9406afa0 0x7f2f9406d460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2fa81648f0 tx=0x7f2f9c007b50 comp rx=0 tx=0).stop 2024-01-19T04:55:28.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa806bc90 msgr2=0x7f2fa8163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 --2- 172.21.15.31:0/3370054163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fa806bc90 0x7f2fa8163380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c480 tx=0x7f2f9800c940 comp rx=0 tx=0).stop 2024-01-19T04:55:28.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fa77fe640 1 -- 172.21.15.31:0/3370054163 reap_dead start 2024-01-19T04:55:28.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 shutdown_connections 2024-01-19T04:55:28.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 >> 172.21.15.31:0/3370054163 conn(0x7f2fa80b1a20 msgr2=0x7f2fa80cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7efd980731c0 msgr2=0x7efd980755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7efdbc741640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7efd980731c0 0x7efd980755e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f2af4073140 msgr2=0x7f2af4075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.542+0000 7f2b1599e640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f2af4073140 0x7f2af4075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 shutdown_connections 2024-01-19T04:55:28.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2fad8ee640 1 -- 172.21.15.31:0/3370054163 wait complete. 2024-01-19T04:55:28.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f7660073140 msgr2=0x7f7660075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f8c68073050 msgr2=0x7f8c68075470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f8c8adcd640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f8c68073050 0x7f8c68075470 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f7660073140 0x7f7660075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2af406afa0 msgr2=0x7f2af406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1599e640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2af406afa0 0x7f2af406d460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b04007fd0 tx=0x7f2b04007e80 comp rx=0 tx=0).stop 2024-01-19T04:55:28.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b080e30b0 msgr2=0x7f2b080debf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1599e640 1 --2- 172.21.15.31:0/3444947729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b080e30b0 0x7f2b080debf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b10073470 tx=0x7f2b1006b990 comp rx=0 tx=0).stop 2024-01-19T04:55:28.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd9806afa0 msgr2=0x7efd9806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd9806afa0 0x7efd9806d460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009aa0 tx=0x7efdb0009190 comp rx=0 tx=0).stop 2024-01-19T04:55:28.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1519d640 1 -- 172.21.15.31:0/3444947729 reap_dead start 2024-01-19T04:55:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb4161930 msgr2=0x7efdb40acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 --2- 172.21.15.31:0/816622369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdb4161930 0x7efdb40acac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdac02c3b0 tx=0x7efdac004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbacde640 1 -- 172.21.15.31:0/816622369 reap_dead start 2024-01-19T04:55:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 shutdown_connections 2024-01-19T04:55:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 >> 172.21.15.31:0/3444947729 conn(0x7f2b080559d0 msgr2=0x7f2b080578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c6806afa0 msgr2=0x7f8c6806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f8c8adcd640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c6806afa0 0x7f8c6806d460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009aa0 tx=0x7f8c74009190 comp rx=0 tx=0).stop 2024-01-19T04:55:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7c0e4cf0 msgr2=0x7f8c7c04f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f8c8adcd640 1 --2- 172.21.15.31:0/2402935440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c7c0e4cf0 0x7f8c7c04f170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c780 tx=0x7f8c8000cc40 comp rx=0 tx=0).stop 2024-01-19T04:55:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 shutdown_connections 2024-01-19T04:55:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 >> 172.21.15.31:0/816622369 conn(0x7efdb406a790 msgr2=0x7efdb40714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f8c8a5cc640 1 -- 172.21.15.31:0/2402935440 reap_dead start 2024-01-19T04:55:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 shutdown_connections 2024-01-19T04:55:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f766006af20 msgr2=0x7f766006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f766006af20 0x7f766006d3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f766c035000 tx=0x7f766c030000 comp rx=0 tx=0).stop 2024-01-19T04:55:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76740cf880 msgr2=0x7f76741532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 --2- 172.21.15.31:0/1841666727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76740cf880 0x7f76741532e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f766800c440 tx=0x7f766800c840 comp rx=0 tx=0).stop 2024-01-19T04:55:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f7678d3b640 1 -- 172.21.15.31:0/1841666727 reap_dead start 2024-01-19T04:55:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 shutdown_connections 2024-01-19T04:55:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 shutdown_connections 2024-01-19T04:55:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 >> 172.21.15.31:0/1841666727 conn(0x7f767406a790 msgr2=0x7f76741565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.543+0000 7efdbc741640 1 -- 172.21.15.31:0/816622369 wait complete. 2024-01-19T04:55:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 shutdown_connections 2024-01-19T04:55:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 shutdown_connections 2024-01-19T04:55:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 >> 172.21.15.31:0/2402935440 conn(0x7f8c7c020000 msgr2=0x7f8c7c009660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f765a7fc640 1 -- 172.21.15.31:0/1841666727 wait complete. 2024-01-19T04:55:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f2b1599e640 1 -- 172.21.15.31:0/3444947729 wait complete. 2024-01-19T04:55:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 shutdown_connections 2024-01-19T04:55:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f8c8adcd640 1 -- 172.21.15.31:0/2402935440 wait complete. 2024-01-19T04:55:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.544+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 --> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f531c0057a0 con 0x7f5308073170 2024-01-19T04:55:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.545+0000 7f53217fa640 1 -- 172.21.15.31:0/3974973614 <== osd.4 v2:172.21.15.137:6814/7950289 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f531c0057a0 con 0x7f5308073170 2024-01-19T04:55:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.545+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f5308073170 msgr2=0x7f5308075590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329844640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f5308073170 0x7f5308075590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f530806af50 msgr2=0x7f530806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329844640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f530806af50 0x7f530806d410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5310047000 tx=0x7f531003f000 comp rx=0 tx=0).stop 2024-01-19T04:55:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f531c03bd10 msgr2=0x7f531c0e5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329844640 1 --2- 172.21.15.31:0/3974973614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f531c03bd10 0x7f531c0e5160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5324069420 tx=0x7f532406b990 comp rx=0 tx=0).stop 2024-01-19T04:55:28.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329043640 1 -- 172.21.15.31:0/3974973614 reap_dead start 2024-01-19T04:55:28.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 shutdown_connections 2024-01-19T04:55:28.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.546+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 >> 172.21.15.31:0/3974973614 conn(0x7f531c020000 msgr2=0x7f531c0095d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.547+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 shutdown_connections 2024-01-19T04:55:28.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.547+0000 7f5329844640 1 -- 172.21.15.31:0/3974973614 wait complete. 2024-01-19T04:55:28.606 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-01-19T04:55:28.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-19T04:55:28.607 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:28.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-19T04:55:28.608 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:28.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-19T04:55:28.609 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-01-19T04:55:28.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-19T04:55:28.610 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:28.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-19T04:55:28.611 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:28.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-19T04:55:28.612 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:28.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-19T04:55:28.613 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:28.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-19T04:55:28.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.817+0000 7f491db68640 1 Processor -- start 2024-01-19T04:55:28.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.817+0000 7f491db68640 1 -- start start 2024-01-19T04:55:28.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.818+0000 7f491db68640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49180cbbc0 0x7f49180cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.818+0000 7f491db68640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180cbfa0 con 0x7f49180cd3d0 2024-01-19T04:55:28.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.818+0000 7f491db68640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180cc930 con 0x7f491806a5b0 2024-01-19T04:55:28.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.818+0000 7f491db68640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180cd810 con 0x7f49180cbbc0 2024-01-19T04:55:28.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7f4917fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f49180cd3d0 0x7f49180af130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56946/0 (socket says 172.21.15.31:56946) 2024-01-19T04:55:28.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7f4917fff640 1 -- 172.21.15.31:0/2794754574 learned_addr learned my addr 172.21.15.31:0/2794754574 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:28.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801243094 0 0) 0x7f49180cbfa0 con 0x7f49180cd3d0 2024-01-19T04:55:28.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7f508df68640 1 Processor -- start 2024-01-19T04:55:28.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7fd91b57c640 1 Processor -- start 2024-01-19T04:55:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4908003520 con 0x7f49180cd3d0 2024-01-19T04:55:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293967384 0 0) 0x7f49180cc930 con 0x7f491806a5b0 2024-01-19T04:55:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49180cbfa0 con 0x7f491806a5b0 2024-01-19T04:55:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1857698043 0 0) 0x7f4908003520 con 0x7f49180cd3d0 2024-01-19T04:55:28.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f491cb66640 1 --2- 172.21.15.31:0/2794754574 >> v2:172.21.15.137:3300/0 conn(0x7f49180cbbc0 0x7f49180cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7f508df68640 1 -- start start 2024-01-19T04:55:28.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 >> v2:172.21.15.137:3300/0 conn(0x7f49180cbbc0 msgr2=0x7f49180cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 --2- 172.21.15.31:0/2794754574 >> v2:172.21.15.137:3300/0 conn(0x7f49180cbbc0 0x7f49180cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 >> v1:172.21.15.137:6789/0 conn(0x7f491806a5b0 legacy=0x7f49180ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180cfa90 con 0x7f49180cd3d0 2024-01-19T04:55:28.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7f38c7778640 1 Processor -- start 2024-01-19T04:55:28.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f38c7778640 1 -- start start 2024-01-19T04:55:28.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.820+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f490c001670 con 0x7f49180cd3d0 2024-01-19T04:55:28.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 >> v1:172.21.15.31:6789/0 conn(0x7f49180cd3d0 legacy=0x7f49180af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f49177fe640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4908003b00 0x7f4908003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f49177fe640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49080047c0 0x7f4908004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f49177fe640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49080051d0 0x7f490800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c7778640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c00b1e60 0x7f38c00b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c7778640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f38c00b2820 0x7f38c016dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c7778640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c0070740 con 0x7f38c0072500 2024-01-19T04:55:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c7778640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c0070560 con 0x7f38c00b2820 2024-01-19T04:55:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c7778640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0070d00 con 0x7f38c00b1e60 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f508df68640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50880cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f508df68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50880cffb0 0x7f50880d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f508df68640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f50880b67b0 0x7f50880b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f508df68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50880b4cc0 con 0x7f50880cffb0 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f508df68640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50880d21f0 con 0x7f50880b67b0 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f508df68640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50880b6320 con 0x7f50880b48e0 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5086ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50880cffb0 0x7f50880d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5086ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50880cffb0 0x7f50880d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36204/0 (socket says 172.21.15.31:36204) 2024-01-19T04:55:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5086ffd640 1 -- 172.21.15.31:0/2219923461 learned_addr learned my addr 172.21.15.31:0/2219923461 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.819+0000 7fd91b57c640 1 -- start start 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91b57c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914165390 0x7fd914165c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91b57c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd914146d30 0x7fd9141471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91b57c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd914161c70 con 0x7fd91406d5b0 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91b57c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd914161f40 con 0x7fd914146d30 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91b57c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd914162220 con 0x7fd914165390 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91ad7b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd914146d30 0x7fd9141471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91ad7b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd914146d30 0x7fd9141471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34974/0 (socket says 172.21.15.31:34974) 2024-01-19T04:55:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7fd91ad7b640 1 -- 172.21.15.31:0/3202390337 learned_addr learned my addr 172.21.15.31:0/3202390337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f4917fff640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49080047c0 0x7f4908004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c4d13640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c00b1e60 0x7f38c00b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c4d13640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c00b1e60 0x7f38c00b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:32866/0 (socket says 172.21.15.31:32866) 2024-01-19T04:55:28.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f38c4d13640 1 -- 172.21.15.31:0/3219619354 learned_addr learned my addr 172.21.15.31:0/3219619354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38c5d15640 1 --2- 172.21.15.31:0/3219619354 >> v2:172.21.15.137:3300/0 conn(0x7f38c00b2820 0x7f38c016dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7fa5e955a640 1 Processor -- start 2024-01-19T04:55:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180cd810 con 0x7f4908003b00 2024-01-19T04:55:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490800c100 con 0x7f49080051d0 2024-01-19T04:55:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7fa5e955a640 1 -- start start 2024-01-19T04:55:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490800c2c0 con 0x7f49080047c0 2024-01-19T04:55:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f491cb66640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4908003b00 0x7f4908003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f491d367640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49080051d0 0x7f490800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f4b610d9640 1 Processor -- start 2024-01-19T04:55:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f4b610d9640 1 -- start start 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f491cb66640 1 -- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49080047c0 msgr2=0x7f4908004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f491cb66640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49080047c0 0x7f4908004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f491cb66640 1 -- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49080051d0 msgr2=0x7f490800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f491cb66640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49080051d0 0x7f490800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f491cb66640 1 -- 172.21.15.31:0/2794754574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4908003520 con 0x7f4908003b00 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7fd91ad7b640 1 -- 172.21.15.31:0/3202390337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914165390 msgr2=0x7fd914165c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7fd91ad7b640 1 --2- 172.21.15.31:0/3202390337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914165390 0x7fd914165c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7fd91ad7b640 1 -- 172.21.15.31:0/3202390337 >> v1:172.21.15.31:6789/0 conn(0x7fd91406d5b0 legacy=0x7fd91406d990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-19T04:55:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7fd91ad7b640 1 -- 172.21.15.31:0/3202390337 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914162500 con 0x7fd914146d30 2024-01-19T04:55:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317195101 0 0) 0x7f38c0070740 con 0x7f38c0072500 2024-01-19T04:55:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38ac003520 con 0x7f38c0072500 2024-01-19T04:55:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 <== mon.1 v1:172.21.15.137:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790838763 0 0) 0x7f38ac003520 con 0x7f38c0072500 2024-01-19T04:55:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f23e04b4640 1 Processor -- start 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c00b1e60 msgr2=0x7f38c00b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c00b1e60 0x7f38c00b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 >> v2:172.21.15.137:3300/0 conn(0x7f38c00b2820 msgr2=0x7f38c016dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 --2- 172.21.15.31:0/3219619354 >> v2:172.21.15.137:3300/0 conn(0x7f38c00b2820 0x7f38c016dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 --> v1:172.21.15.137:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c0070a20 con 0x7f38c0072500 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 <== mon.1 v1:172.21.15.137:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f38bc001670 con 0x7f38c0072500 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 >> v1:172.21.15.137:6789/0 conn(0x7f38c0072500 legacy=0x7f38c006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38ac003970 0x7f38ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ac004360 0x7f38ac004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38ac004f60 0x7f38ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0070560 con 0x7f38ac004360 2024-01-19T04:55:28.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0070d00 con 0x7f38ac004f60 2024-01-19T04:55:28.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac01cf00 con 0x7f38ac003970 2024-01-19T04:55:28.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f38c5514640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38ac003970 0x7f38ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f491d367640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49080051d0 0x7f490800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:28.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5087fff640 1 --2- 172.21.15.31:0/2219923461 >> v2:172.21.15.137:3300/0 conn(0x7f50880b67b0 0x7f50880b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f50877fe640 1 --2- 172.21.15.31:0/2219923461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50880cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e955a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e413e0e0 0x7fa5e40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e955a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5e40cdda0 0x7fa5e41408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5086ffd640 1 -- 172.21.15.31:0/2219923461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 msgr2=0x7f50880cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5086ffd640 1 --2- 172.21.15.31:0/2219923461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50880cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5086ffd640 1 -- 172.21.15.31:0/2219923461 >> v2:172.21.15.137:3300/0 conn(0x7f50880b67b0 msgr2=0x7f50880b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.821+0000 7f5086ffd640 1 --2- 172.21.15.31:0/2219923461 >> v2:172.21.15.137:3300/0 conn(0x7f50880b67b0 0x7f50880b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.822+0000 7f5086ffd640 1 -- 172.21.15.31:0/2219923461 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880d0c30 con 0x7f50880cffb0 2024-01-19T04:55:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f5086ffd640 1 --2- 172.21.15.31:0/2219923461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 0x7f50880d0430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f508808bcc0 tx=0x7f508002bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f119e6d9f40d73 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e955a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40ceca0 con 0x7fa5e4140e20 2024-01-19T04:55:28.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e955a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e40ce200 con 0x7fa5e40cdda0 2024-01-19T04:55:28.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e955a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41412b0 con 0x7fa5e413e0e0 2024-01-19T04:55:28.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f491d367640 1 -- 172.21.15.31:0/2794754574 reap_dead start 2024-01-19T04:55:28.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f50867fc640 1 -- 172.21.15.31:0/2219923461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508002c980 con 0x7f50880cffb0 2024-01-19T04:55:28.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f4917fff640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49080047c0 0x7f4908004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:28.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e2ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e413e0e0 0x7fa5e40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e2ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e413e0e0 0x7fa5e40cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:32880/0 (socket says 172.21.15.31:32880) 2024-01-19T04:55:28.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7fa5e2ffd640 1 -- 172.21.15.31:0/2913799469 learned_addr learned my addr 172.21.15.31:0/2913799469 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f50867fc640 1 -- 172.21.15.31:0/2219923461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508002cae0 con 0x7f50880cffb0 2024-01-19T04:55:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.823+0000 7f23e04b4640 1 -- start start 2024-01-19T04:55:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f38c5d15640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38ac004f60 0x7f38ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fa5e27fc640 1 --2- 172.21.15.31:0/2913799469 >> v2:172.21.15.31:3300/0 conn(0x7fa5e40cdda0 0x7fa5e41408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f4b610d9640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4b5c159460 0x7f4b5c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f38c4d13640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ac004360 0x7f38ac004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fd91ad7b640 1 --2- 172.21.15.31:0/3202390337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914146d30 0x7fd9141471c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009e50 tx=0x7fd90c02bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b096e30041c9a1f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f4b610d9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fa5e1ffb640 1 -- 172.21.15.31:0/2913799469 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722117543 0 0) 0x7fa5e40ceca0 con 0x7fa5e4140e20 2024-01-19T04:55:28.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fa5e1ffb640 1 -- 172.21.15.31:0/2913799469 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5cc003520 con 0x7fa5e4140e20 2024-01-19T04:55:28.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f4b610d9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b5c06daf0 0x7f4b5c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f4b5bfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4b5c159460 0x7f4b5c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f4b5bfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4b5c159460 0x7f4b5c15d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35008/0 (socket says 172.21.15.31:35008) 2024-01-19T04:55:28.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fa5e2ffd640 1 -- 172.21.15.31:0/2913799469 >> v2:172.21.15.31:3300/0 conn(0x7fa5e40cdda0 msgr2=0x7fa5e41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fa5e2ffd640 1 --2- 172.21.15.31:0/2913799469 >> v2:172.21.15.31:3300/0 conn(0x7fa5e40cdda0 0x7fa5e41408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fa5e2ffd640 1 -- 172.21.15.31:0/2913799469 >> v1:172.21.15.137:6789/0 conn(0x7fa5e4140e20 legacy=0x7fa5e415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fa5e2ffd640 1 -- 172.21.15.31:0/2913799469 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40b2320 con 0x7fa5e413e0e0 2024-01-19T04:55:28.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f4b5bfff640 1 -- 172.21.15.31:0/3726021940 learned_addr learned my addr 172.21.15.31:0/3726021940 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f069db97640 1 Processor -- start 2024-01-19T04:55:28.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f069db97640 1 -- start start 2024-01-19T04:55:28.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fd919578640 1 -- 172.21.15.31:0/3202390337 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c02caa0 con 0x7fd914146d30 2024-01-19T04:55:28.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7fd919578640 1 -- 172.21.15.31:0/3202390337 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c02cc00 con 0x7fd914146d30 2024-01-19T04:55:28.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f491cb66640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4908003b00 0x7f4908003ee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f490400a970 tx=0x7f49080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90a283d2f89e546 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23e04b4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f23d80ca860 0x7f23d8071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f508df68640 1 -- 172.21.15.31:0/2219923461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 msgr2=0x7f50880d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f508df68640 1 --2- 172.21.15.31:0/2219923461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 0x7f50880d0430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f508808bcc0 tx=0x7f508002bde0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f38c4d13640 1 -- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38ac003970 msgr2=0x7f38ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f38c4d13640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38ac003970 0x7f38ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f38c4d13640 1 -- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38ac004f60 msgr2=0x7f38ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f38c4d13640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38ac004f60 0x7f38ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f38c4d13640 1 -- 172.21.15.31:0/3219619354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c0070a20 con 0x7f38ac004360 2024-01-19T04:55:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f508df68640 1 -- 172.21.15.31:0/2219923461 shutdown_connections 2024-01-19T04:55:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f508df68640 1 --2- 172.21.15.31:0/2219923461 >> v2:172.21.15.137:3300/0 conn(0x7f50880b67b0 0x7f50880b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f508df68640 1 --2- 172.21.15.31:0/2219923461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 0x7f50880d0430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f508df68640 1 --2- 172.21.15.31:0/2219923461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50880cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f508df68640 1 -- 172.21.15.31:0/2219923461 >> 172.21.15.31:0/2219923461 conn(0x7f50880d3310 msgr2=0x7f50880d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904031680 con 0x7f4908003b00 2024-01-19T04:55:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23de250640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f23d80ca860 0x7f23d8071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23de250640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f23d80ca860 0x7f23d8071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35016/0 (socket says 172.21.15.31:35016) 2024-01-19T04:55:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23dda4f640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f23d815a3a0 0x7f23d80cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:46434/0 (socket says 172.21.15.31:46434) 2024-01-19T04:55:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23e04b4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f23d80cb4c0 0x7f23d806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23de250640 1 -- 172.21.15.31:0/502887228 learned_addr learned my addr 172.21.15.31:0/502887228 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d80ce380 con 0x7f23d815a3a0 2024-01-19T04:55:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f508df68640 1 -- 172.21.15.31:0/2219923461 shutdown_connections 2024-01-19T04:55:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d8072670 con 0x7f23d80cb4c0 2024-01-19T04:55:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d80cdd70 con 0x7f23d80ca860 2024-01-19T04:55:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.824+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/3726021940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b608d8640 1 --2- 172.21.15.31:0/3726021940 >> v2:172.21.15.31:3300/0 conn(0x7f4b5c06daf0 0x7f4b5c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c159840 con 0x7f4b5c06daf0 2024-01-19T04:55:28.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c06df50 con 0x7f4b5c159460 2024-01-19T04:55:28.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c0cd120 con 0x7f4b5c070330 2024-01-19T04:55:28.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23dd24e640 1 -- 172.21.15.31:0/502887228 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693646756 0 0) 0x7f23d80ce380 con 0x7f23d815a3a0 2024-01-19T04:55:28.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23dd24e640 1 -- 172.21.15.31:0/502887228 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c4003520 con 0x7f23d815a3a0 2024-01-19T04:55:28.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b5b7fe640 1 -- 172.21.15.31:0/3726021940 >> v2:172.21.15.137:3300/0 conn(0x7f4b5c159460 msgr2=0x7f4b5c15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23dea51640 1 --2- 172.21.15.31:0/502887228 >> v2:172.21.15.31:3300/0 conn(0x7f23d80cb4c0 0x7f23d806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23de250640 1 -- 172.21.15.31:0/502887228 >> v2:172.21.15.31:3300/0 conn(0x7f23d80cb4c0 msgr2=0x7f23d806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7fd91b57c640 1 -- 172.21.15.31:0/3202390337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914146d30 msgr2=0x7fd9141471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7fd91b57c640 1 --2- 172.21.15.31:0/3202390337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914146d30 0x7fd9141471c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009e50 tx=0x7fd90c02bf00 comp rx=0 tx=0).stop 2024-01-19T04:55:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f069db97640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f069815d5c0 0x7f06980d3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f069db97640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f06980cdec0 0x7f06980d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7fd91b57c640 1 -- 172.21.15.31:0/3202390337 shutdown_connections 2024-01-19T04:55:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7fd91b57c640 1 --2- 172.21.15.31:0/3202390337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914146d30 0x7fd9141471c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7fd91b57c640 1 --2- 172.21.15.31:0/3202390337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914165390 0x7fd914165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7fd91b57c640 1 -- 172.21.15.31:0/3202390337 >> 172.21.15.31:0/3202390337 conn(0x7fd9140b0bd0 msgr2=0x7fd9140b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49040317e0 con 0x7f4908003b00 2024-01-19T04:55:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/3726021940 >> v2:172.21.15.137:3300/0 conn(0x7f4b5c159460 0x7f4b5c15d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b5b7fe640 1 -- 172.21.15.31:0/3726021940 >> v2:172.21.15.31:3300/0 conn(0x7f4b5c06daf0 msgr2=0x7f4b5c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/3726021940 >> v2:172.21.15.31:3300/0 conn(0x7f4b5c06daf0 0x7f4b5c0cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f4b5b7fe640 1 -- 172.21.15.31:0/3726021940 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c15de70 con 0x7f4b5c070330 2024-01-19T04:55:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7fd91b57c640 1 -- 172.21.15.31:0/3202390337 shutdown_connections 2024-01-19T04:55:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7fd91b57c640 1 -- 172.21.15.31:0/3202390337 wait complete. 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f06977fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f069815d5c0 0x7f06980d3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f0696ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f06980cdec0 0x7f06980d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f069db97640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06980d1450 con 0x7f06980d1e40 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f06977fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f069815d5c0 0x7f06980d3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35026/0 (socket says 172.21.15.31:35026) 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f06977fe640 1 -- 172.21.15.31:0/2034477614 learned_addr learned my addr 172.21.15.31:0/2034477614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f0696ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f06980cdec0 0x7f06980d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:32904/0 (socket says 172.21.15.31:32904) 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f069db97640 1 -- 172.21.15.31:0/2034477614 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06980d22d0 con 0x7f069815d5c0 2024-01-19T04:55:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f069db97640 1 -- 172.21.15.31:0/2034477614 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06980cf240 con 0x7f06980cdec0 2024-01-19T04:55:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f49177fe640 1 -- 172.21.15.31:0/2794754574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904031c10 con 0x7f4908003b00 2024-01-19T04:55:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23de250640 1 --2- 172.21.15.31:0/502887228 >> v2:172.21.15.31:3300/0 conn(0x7f23d80cb4c0 0x7f23d806a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23de250640 1 -- 172.21.15.31:0/502887228 >> v1:172.21.15.137:6789/0 conn(0x7f23d815a3a0 legacy=0x7f23d80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.825+0000 7f23de250640 1 -- 172.21.15.31:0/502887228 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8072390 con 0x7f23d80ca860 2024-01-19T04:55:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f23dea51640 1 --2- 172.21.15.31:0/502887228 >> v2:172.21.15.31:3300/0 conn(0x7f23d80cb4c0 0x7f23d806a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7fa5e2ffd640 1 --2- 172.21.15.31:0/2913799469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e413e0e0 0x7fa5e40cd860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009870 tx=0x7fa5d402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e42b534e203e79b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7fa5e1ffb640 1 -- 172.21.15.31:0/2913799469 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4031640 con 0x7fa5e413e0e0 2024-01-19T04:55:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7fa5e1ffb640 1 -- 172.21.15.31:0/2913799469 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d40317a0 con 0x7fa5e413e0e0 2024-01-19T04:55:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f491db68640 1 -- 172.21.15.31:0/2794754574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4908003b00 msgr2=0x7f4908003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f491db68640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4908003b00 0x7f4908003ee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f490400a970 tx=0x7f49080057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/3726021940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c06d5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c06beb0 tx=0x7f4b5002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad42027fdc5ef7e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f491db68640 1 -- 172.21.15.31:0/2794754574 shutdown_connections 2024-01-19T04:55:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f491db68640 1 --2- 172.21.15.31:0/2794754574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4908003b00 0x7f4908003ee0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f490400a970 tx=0x7f49080057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f491db68640 1 -- 172.21.15.31:0/2794754574 >> 172.21.15.31:0/2794754574 conn(0x7f49180b1a40 msgr2=0x7f49180b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f38c4d13640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ac004360 0x7f38ac004840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009ab0 tx=0x7f38ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b62ec5509882d50 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f0696ffd640 1 -- 172.21.15.31:0/2034477614 >> v2:172.21.15.137:3300/0 conn(0x7f069815d5c0 msgr2=0x7f06980d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f0696ffd640 1 --2- 172.21.15.31:0/2034477614 >> v2:172.21.15.137:3300/0 conn(0x7f069815d5c0 0x7f06980d3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f0696ffd640 1 -- 172.21.15.31:0/2034477614 >> v1:172.21.15.31:6789/0 conn(0x7f06980d1e40 legacy=0x7f06980cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f0696ffd640 1 -- 172.21.15.31:0/2034477614 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980ce6c0 con 0x7f06980cdec0 2024-01-19T04:55:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.826+0000 7f508df68640 1 -- 172.21.15.31:0/2219923461 wait complete. 2024-01-19T04:55:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f23de250640 1 --2- 172.21.15.31:0/502887228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d8071d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23d8074250 tx=0x7f23c802fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffcc7d2b952d8b3e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7fa5e1ffb640 1 -- 172.21.15.31:0/2913799469 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4031b30 con 0x7fa5e413e0e0 2024-01-19T04:55:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f4b5affd640 1 -- 172.21.15.31:0/3726021940 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50031780 con 0x7f4b5c070330 2024-01-19T04:55:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f4b5affd640 1 -- 172.21.15.31:0/3726021940 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b500318e0 con 0x7f4b5c070330 2024-01-19T04:55:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f4b5affd640 1 -- 172.21.15.31:0/3726021940 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50031c70 con 0x7f4b5c070330 2024-01-19T04:55:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b002c600 con 0x7f38ac004360 2024-01-19T04:55:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b002c760 con 0x7f38ac004360 2024-01-19T04:55:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f491db68640 1 -- 172.21.15.31:0/2794754574 shutdown_connections 2024-01-19T04:55:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f491db68640 1 -- 172.21.15.31:0/2794754574 wait complete. 2024-01-19T04:55:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f23dd24e640 1 -- 172.21.15.31:0/502887228 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8030960 con 0x7f23d80ca860 2024-01-19T04:55:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f23dd24e640 1 -- 172.21.15.31:0/502887228 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c8030ac0 con 0x7f23d80ca860 2024-01-19T04:55:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7f23dd24e640 1 -- 172.21.15.31:0/502887228 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8030e50 con 0x7f23d80ca860 2024-01-19T04:55:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7fd91b57c640 1 Processor -- start 2024-01-19T04:55:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7fd91b57c640 1 -- start start 2024-01-19T04:55:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7fa5e955a640 1 -- 172.21.15.31:0/2913799469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e413e0e0 msgr2=0x7fa5e40cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.827+0000 7fa5e955a640 1 --2- 172.21.15.31:0/2913799469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e413e0e0 0x7fa5e40cd860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009870 tx=0x7fa5d402b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7fa5e955a640 1 -- 172.21.15.31:0/2913799469 shutdown_connections 2024-01-19T04:55:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7fa5e955a640 1 --2- 172.21.15.31:0/2913799469 >> v2:172.21.15.31:3300/0 conn(0x7fa5e40cdda0 0x7fa5e41408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7fa5e955a640 1 --2- 172.21.15.31:0/2913799469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e413e0e0 0x7fa5e40cd860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7fa5e955a640 1 -- 172.21.15.31:0/2913799469 >> 172.21.15.31:0/2913799469 conn(0x7fa5e40b0b60 msgr2=0x7fa5e40b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7fa5e955a640 1 -- 172.21.15.31:0/2913799469 shutdown_connections 2024-01-19T04:55:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38b7fff640 1 -- 172.21.15.31:0/3219619354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b002cb10 con 0x7f38ac004360 2024-01-19T04:55:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7fa5e955a640 1 -- 172.21.15.31:0/2913799469 wait complete. 2024-01-19T04:55:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 msgr2=0x7f4b5c06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 --2- 172.21.15.31:0/3726021940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c06d5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c06beb0 tx=0x7f4b5002b820 comp rx=0 tx=0).stop 2024-01-19T04:55:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38c7778640 1 -- 172.21.15.31:0/3219619354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ac004360 msgr2=0x7f38ac004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38c7778640 1 --2- 172.21.15.31:0/3219619354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ac004360 0x7f38ac004840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009ab0 tx=0x7f38ac0057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38c5d15640 1 -- 172.21.15.31:0/3219619354 reap_dead start 2024-01-19T04:55:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38c7778640 1 -- 172.21.15.31:0/3219619354 shutdown_connections 2024-01-19T04:55:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38c7778640 1 -- 172.21.15.31:0/3219619354 >> 172.21.15.31:0/3219619354 conn(0x7f38c00b0b60 msgr2=0x7f38c00b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 msgr2=0x7f23d8071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f23e04b4640 1 --2- 172.21.15.31:0/502887228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d8071d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23d8074250 tx=0x7f23c802fdc0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f508df68640 1 Processor -- start 2024-01-19T04:55:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f508df68640 1 -- start start 2024-01-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 shutdown_connections 2024-01-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 --2- 172.21.15.31:0/3726021940 >> v2:172.21.15.31:3300/0 conn(0x7f4b5c06daf0 0x7f4b5c0cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 --2- 172.21.15.31:0/3726021940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c06d5b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 --2- 172.21.15.31:0/3726021940 >> v2:172.21.15.137:3300/0 conn(0x7f4b5c159460 0x7f4b5c15d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 >> 172.21.15.31:0/3726021940 conn(0x7f4b5c0b0bd0 msgr2=0x7f4b5c0b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38c7778640 1 -- 172.21.15.31:0/3219619354 shutdown_connections 2024-01-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 shutdown_connections 2024-01-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f4b610d9640 1 -- 172.21.15.31:0/3726021940 wait complete. 2024-01-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.828+0000 7f38c7778640 1 -- 172.21.15.31:0/3219619354 wait complete. 2024-01-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 shutdown_connections 2024-01-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f23e04b4640 1 --2- 172.21.15.31:0/502887228 >> v2:172.21.15.31:3300/0 conn(0x7f23d80cb4c0 0x7f23d806a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f23e04b4640 1 --2- 172.21.15.31:0/502887228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d8071d80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 >> 172.21.15.31:0/502887228 conn(0x7f23d80d0000 msgr2=0x7f23d80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f0696ffd640 1 --2- 172.21.15.31:0/2034477614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f06980cdec0 0x7f06980d1900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f068c0097a0 tx=0x7f068c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b35010bbce424a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91b57c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91406d5b0 0x7fd914163790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91b57c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914146d30 0x7fd9140b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91b57c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914165390 0x7fd91415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91b57c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141650e0 con 0x7fd91406d5b0 2024-01-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91b57c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b0730 con 0x7fd914165390 2024-01-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91b57c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd914163d70 con 0x7fd914146d30 2024-01-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f508df68640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50881622a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f491db68640 1 Processor -- start 2024-01-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91406d5b0 0x7fd914163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91406d5b0 0x7fd914163790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36240/0 (socket says 172.21.15.31:36240) 2024-01-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 -- 172.21.15.31:0/4019950649 learned_addr learned my addr 172.21.15.31:0/4019950649 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f491db68640 1 -- start start 2024-01-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd919d79640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914146d30 0x7fd9140b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f508df68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50880b67b0 0x7f508815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f508df68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 0x7f50881f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f508df68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50881f8450 con 0x7f50880cffb0 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f508df68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5088162940 con 0x7f50880b67b0 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f508df68640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5088162c20 con 0x7f50880b48e0 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f50877fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50881622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f50877fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50881622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:32934/0 (socket says 172.21.15.31:32934) 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f50877fe640 1 -- 172.21.15.31:0/3667861324 learned_addr learned my addr 172.21.15.31:0/3667861324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f5086ffd640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50880b67b0 0x7f508815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91ad7b640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914165390 0x7fd91415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7f5087fff640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 0x7f50881f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 -- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914146d30 msgr2=0x7fd9140b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914146d30 0x7fd9140b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 -- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914165390 msgr2=0x7fd91415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914165390 0x7fd91415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.829+0000 7fd91a57a640 1 -- 172.21.15.31:0/4019950649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c009b30 con 0x7fd91406d5b0 2024-01-19T04:55:28.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7fd919d79640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914146d30 0x7fd9140b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:28.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 shutdown_connections 2024-01-19T04:55:28.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f06967fc640 1 -- 172.21.15.31:0/2034477614 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c031780 con 0x7f06980cdec0 2024-01-19T04:55:28.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f5086ffd640 1 -- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 msgr2=0x7f50881622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f06967fc640 1 -- 172.21.15.31:0/2034477614 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c0318e0 con 0x7f06980cdec0 2024-01-19T04:55:28.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f5086ffd640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50881622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f5086ffd640 1 -- 172.21.15.31:0/3667861324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 msgr2=0x7f50881f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f5086ffd640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 0x7f50881f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f06967fc640 1 -- 172.21.15.31:0/2034477614 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c031c70 con 0x7f06980cdec0 2024-01-19T04:55:28.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f5086ffd640 1 -- 172.21.15.31:0/3667861324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5080009b30 con 0x7f50880b67b0 2024-01-19T04:55:28.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7fd91a57a640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91406d5b0 0x7fd914163790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b450 tx=0x7fd91000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7fa5e955a640 1 Processor -- start 2024-01-19T04:55:28.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f5086ffd640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50880b67b0 0x7f508815eb80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5080032000 tx=0x7f5080004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7fa5e955a640 1 -- start start 2024-01-19T04:55:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f506ffff640 1 -- 172.21.15.31:0/3667861324 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508003f640 con 0x7f50880b67b0 2024-01-19T04:55:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f506ffff640 1 -- 172.21.15.31:0/3667861324 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508003f7c0 con 0x7f50880b67b0 2024-01-19T04:55:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f491db68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49181f49e0 0x7f49180b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f491db68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49180b3e90 0x7f49181f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f491db68640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49180c3ad0 0x7f49181f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f4b610d9640 1 Processor -- start 2024-01-19T04:55:28.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f491db68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180b4fc0 con 0x7f49180b3e90 2024-01-19T04:55:28.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f491db68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49181f8fe0 con 0x7f49181f49e0 2024-01-19T04:55:28.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e955a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e40cdda0 0x7fa5e41f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e955a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e413e0e0 0x7fa5e413c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e955a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e4140e20 0x7fa5e40af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e955a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41f7070 con 0x7fa5e413e0e0 2024-01-19T04:55:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e955a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e413cea0 con 0x7fa5e4140e20 2024-01-19T04:55:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e955a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e413d180 con 0x7fa5e40cdda0 2024-01-19T04:55:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f4b610d9640 1 -- start start 2024-01-19T04:55:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f069db97640 1 -- 172.21.15.31:0/2034477614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f06980cdec0 msgr2=0x7f06980d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f069db97640 1 --2- 172.21.15.31:0/2034477614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f06980cdec0 0x7f06980d1900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f068c0097a0 tx=0x7f068c02b820 comp rx=0 tx=0).stop 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e413e0e0 0x7fa5e413c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e2ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e40cdda0 0x7fa5e41f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e413e0e0 0x7fa5e413c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36264/0 (socket says 172.21.15.31:36264) 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7fa5e27fc640 1 -- 172.21.15.31:0/15004949 learned_addr learned my addr 172.21.15.31:0/15004949 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f38c7778640 1 Processor -- start 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f38c7778640 1 -- start start 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f38c7778640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c0089b40 0x7f38c00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f38c7778640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38c0162020 0x7f38c01682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f38c7778640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c0168810 0x7f38c01635d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f38c7778640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0168cc0 con 0x7f38c0168810 2024-01-19T04:55:28.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f38c7778640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0089f20 con 0x7f38c0162020 2024-01-19T04:55:28.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f38c7778640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00ccaf0 con 0x7f38c0089b40 2024-01-19T04:55:28.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f069db97640 1 -- 172.21.15.31:0/2034477614 shutdown_connections 2024-01-19T04:55:28.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f069db97640 1 --2- 172.21.15.31:0/2034477614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f06980cdec0 0x7f06980d1900 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f069db97640 1 --2- 172.21.15.31:0/2034477614 >> v2:172.21.15.137:3300/0 conn(0x7f069815d5c0 0x7f06980d3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f069db97640 1 -- 172.21.15.31:0/2034477614 >> 172.21.15.31:0/2034477614 conn(0x7f069809c600 msgr2=0x7f06980d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:28.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fa5e2ffd640 1 -- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e4140e20 msgr2=0x7fa5e40af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c0168810 0x7f38c01635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c0168810 0x7f38c01635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36276/0 (socket says 172.21.15.31:36276) 2024-01-19T04:55:28.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 -- 172.21.15.31:0/3193726951 learned_addr learned my addr 172.21.15.31:0/3193726951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fa5e2ffd640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e4140e20 0x7fa5e40af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c4d13640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38c0162020 0x7f38c01682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5514640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c0089b40 0x7f38c00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fa5e2ffd640 1 -- 172.21.15.31:0/15004949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e413e0e0 msgr2=0x7fa5e413c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fa5e2ffd640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e413e0e0 0x7fa5e413c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fa5e2ffd640 1 -- 172.21.15.31:0/15004949 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40094d0 con 0x7fa5e40cdda0 2024-01-19T04:55:28.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50881f8680 con 0x7f50880b67b0 2024-01-19T04:55:28.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.830+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5088162f00 con 0x7f50880b67b0 2024-01-19T04:55:28.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f506ffff640 1 -- 172.21.15.31:0/3667861324 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508003fb70 con 0x7f50880b67b0 2024-01-19T04:55:28.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f491db68640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49181f4dc0 con 0x7f49180c3ad0 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f491cb66640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49181f49e0 0x7f49180b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f491cb66640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49181f49e0 0x7f49180b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35068/0 (socket says 172.21.15.31:35068) 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f491cb66640 1 -- 172.21.15.31:0/2215460726 learned_addr learned my addr 172.21.15.31:0/2215460726 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f4917fff640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49180b3e90 0x7f49181f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f491d367640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49180c3ad0 0x7f49181f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 -- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c0089b40 msgr2=0x7f38c00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c0089b40 0x7f38c00cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 -- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38c0162020 msgr2=0x7f38c01682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38c0162020 0x7f38c01682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 -- 172.21.15.31:0/3193726951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00cba50 con 0x7f38c0168810 2024-01-19T04:55:28.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.831+0000 7f23e04b4640 1 -- 172.21.15.31:0/502887228 wait complete. 2024-01-19T04:55:28.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f4b610d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b5c06daf0 0x7f4b5c14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fa5e2ffd640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e40cdda0 0x7fa5e41f6b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4033000 tx=0x7fa5d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fd90affd640 1 -- 172.21.15.31:0/4019950649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910011890 con 0x7fd91406d5b0 2024-01-19T04:55:28.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fd90affd640 1 -- 172.21.15.31:0/4019950649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910011a10 con 0x7fd91406d5b0 2024-01-19T04:55:28.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7fd90affd640 1 -- 172.21.15.31:0/4019950649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910011dc0 con 0x7fd91406d5b0 2024-01-19T04:55:28.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f506ffff640 1 -- 172.21.15.31:0/3667861324 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f508003fcd0 con 0x7f50880b67b0 2024-01-19T04:55:28.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd914164050 con 0x7fd91406d5b0 2024-01-19T04:55:28.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f38c5d15640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c0168810 0x7f38c01635d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38c00728e0 tx=0x7f38b8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f491cb66640 1 -- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49180c3ad0 msgr2=0x7f49181f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.832+0000 7f491cb66640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49180c3ad0 0x7f49181f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f491cb66640 1 -- 172.21.15.31:0/2215460726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49180b3e90 msgr2=0x7f49181f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f491cb66640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49180b3e90 0x7f49181f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f491cb66640 1 -- 172.21.15.31:0/2215460726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180b4340 con 0x7f49181f49e0 2024-01-19T04:55:28.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f506dffb640 1 -- 172.21.15.31:0/3667861324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50881f8ba0 con 0x7f50880b67b0 2024-01-19T04:55:28.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd914159700 con 0x7fd91406d5b0 2024-01-19T04:55:28.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f069db97640 1 -- 172.21.15.31:0/2034477614 shutdown_connections 2024-01-19T04:55:28.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f38b5ffb640 1 -- 172.21.15.31:0/3193726951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b80046a0 con 0x7f38c0168810 2024-01-19T04:55:28.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b5c06daf0 0x7f4b5c14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f491cb66640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49181f49e0 0x7f49180b53e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f490400a120 tx=0x7f4904004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f069db97640 1 -- 172.21.15.31:0/2034477614 wait complete. 2024-01-19T04:55:28.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f506ffff640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f507406af50 0x7f507406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b610d9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c1ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b610d9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c159460 0x7f4b5c152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b610d9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c15b760 con 0x7f4b5c06daf0 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b5c06daf0 0x7f4b5c14ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36298/0 (socket says 172.21.15.31:36298) 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b5bfff640 1 -- 172.21.15.31:0/2555989826 learned_addr learned my addr 172.21.15.31:0/2555989826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1eeba0 con 0x7f4b5c159460 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c14b580 con 0x7f4b5c070330 2024-01-19T04:55:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f50877fe640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f507406af50 0x7f507406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f506ffff640 1 -- 172.21.15.31:0/3667861324 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50800ac110 con 0x7f50880b67b0 2024-01-19T04:55:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b608d8640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c159460 0x7f4b5c152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c1ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7fa5cb7fe640 1 -- 172.21.15.31:0/15004949 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d403a9f0 con 0x7fa5e40cdda0 2024-01-19T04:55:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f4b5bfff640 1 -- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 msgr2=0x7f4b5c1ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f4b5bfff640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c1ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f4b5bfff640 1 -- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c159460 msgr2=0x7f4b5c152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f23e04b4640 1 Processor -- start 2024-01-19T04:55:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f4b5bfff640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c159460 0x7f4b5c152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f23e04b4640 1 -- start start 2024-01-19T04:55:28.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f4b5bfff640 1 -- 172.21.15.31:0/2555989826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50009400 con 0x7f4b5c06daf0 2024-01-19T04:55:28.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f50877fe640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f507406af50 0x7f507406d410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5078006fd0 tx=0x7f5078008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c1ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c00cbd30 con 0x7f38c0168810 2024-01-19T04:55:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.833+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c00cc2f0 con 0x7f38c0168810 2024-01-19T04:55:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f4b5bfff640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b5c06daf0 0x7f4b5c14ae80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00e840 tx=0x7f4b4c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7fa5cb7fe640 1 -- 172.21.15.31:0/15004949 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d403ab70 con 0x7fa5e40cdda0 2024-01-19T04:55:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7fa5cb7fe640 1 -- 172.21.15.31:0/15004949 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40443f0 con 0x7fa5e40cdda0 2024-01-19T04:55:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7f38b5ffb640 1 -- 172.21.15.31:0/3193726951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b8004800 con 0x7f38c0168810 2024-01-19T04:55:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9141641e0 con 0x7fd91406d5b0 2024-01-19T04:55:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e413d460 con 0x7fa5e40cdda0 2024-01-19T04:55:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.834+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e4155860 con 0x7fa5e40cdda0 2024-01-19T04:55:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f38b5ffb640 1 -- 172.21.15.31:0/3193726951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8015620 con 0x7f38c0168810 2024-01-19T04:55:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f23e04b4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f069db97640 1 Processor -- start 2024-01-19T04:55:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f23e04b4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23d80cb4c0 0x7f23d81540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f23e04b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d815a3a0 0x7f23d815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a0005290 con 0x7f38c0168810 2024-01-19T04:55:28.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f069db97640 1 -- start start 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f23e04b4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d815d680 con 0x7f23d815a3a0 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.835+0000 7f23de250640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d806d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35100/0 (socket says 172.21.15.31:35100) 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 -- 172.21.15.31:0/301204424 learned_addr learned my addr 172.21.15.31:0/301204424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23e04b4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80c6360 con 0x7f23d80ca860 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80c67f0 con 0x7f23d80cb4c0 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23dda4f640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23d80cb4c0 0x7f23d81540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e4074900 con 0x7fa5e40cdda0 2024-01-19T04:55:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f4b58ff9640 1 -- 172.21.15.31:0/2555989826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00c8e0 con 0x7f4b5c06daf0 2024-01-19T04:55:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f4b58ff9640 1 -- 172.21.15.31:0/2555989826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00ca40 con 0x7f4b5c06daf0 2024-01-19T04:55:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f4b58ff9640 1 -- 172.21.15.31:0/2555989826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00cdb0 con 0x7f4b5c06daf0 2024-01-19T04:55:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c14b860 con 0x7f4b5c06daf0 2024-01-19T04:55:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c14be20 con 0x7f4b5c06daf0 2024-01-19T04:55:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f38b5ffb640 1 -- 172.21.15.31:0/3193726951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f38b8015780 con 0x7f38c0168810 2024-01-19T04:55:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f38b5ffb640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f38a406af00 0x7f38a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23dea51640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d815a3a0 0x7f23d815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 -- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23d80cb4c0 msgr2=0x7f23d81540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23d80cb4c0 0x7f23d81540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 -- 172.21.15.31:0/301204424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d815a3a0 msgr2=0x7f23d815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d815a3a0 0x7f23d815c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 -- 172.21.15.31:0/301204424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c800d970 con 0x7f23d80ca860 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7f23de250640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d806d840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23c803a040 tx=0x7f23c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f069db97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06980cdec0 0x7f06981f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f069db97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06980d1e40 0x7f06981564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f069db97640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f069815d5c0 0x7f06980cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f069db97640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069815ede0 con 0x7f06980cdec0 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f069db97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069815f0c0 con 0x7f06980d1e40 2024-01-19T04:55:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f069db97640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069815f3a0 con 0x7f069815d5c0 2024-01-19T04:55:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7fd90affd640 1 -- 172.21.15.31:0/4019950649 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd91001a400 con 0x7fd91406d5b0 2024-01-19T04:55:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.836+0000 7fd90affd640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8f806af20 0x7fd8f806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7fd90affd640 1 -- 172.21.15.31:0/4019950649 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd910089930 con 0x7fd91406d5b0 2024-01-19T04:55:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f38c5514640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f38a406af00 0x7f38a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f38b5ffb640 1 -- 172.21.15.31:0/3193726951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38b8058830 con 0x7f38c0168810 2024-01-19T04:55:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f0696ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06980d1e40 0x7f06981564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f0696ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06980d1e40 0x7f06981564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35114/0 (socket says 172.21.15.31:35114) 2024-01-19T04:55:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f0696ffd640 1 -- 172.21.15.31:0/3679485381 learned_addr learned my addr 172.21.15.31:0/3679485381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7fd919d79640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8f806af20 0x7fd8f806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f06977fe640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06980cdec0 0x7f06981f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f0697fff640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f069815d5c0 0x7f06980cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7fa5cb7fe640 1 -- 172.21.15.31:0/15004949 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa5d402f050 con 0x7fa5e40cdda0 2024-01-19T04:55:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7fa5cb7fe640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d006afa0 0x7fa5d006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7fa5cb7fe640 1 -- 172.21.15.31:0/15004949 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5d40ab4f0 con 0x7fa5e40cdda0 2024-01-19T04:55:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7fa5e27fc640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d006afa0 0x7fa5d006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.837+0000 7f06977fe640 1 -- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f069815d5c0 msgr2=0x7f06980cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f06977fe640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f069815d5c0 0x7f06980cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f06977fe640 1 -- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06980d1e40 msgr2=0x7f06981564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f06977fe640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06980d1e40 0x7f06981564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f06977fe640 1 -- 172.21.15.31:0/3679485381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c009400 con 0x7f06980cdec0 2024-01-19T04:55:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c1eedc0 con 0x7f4b5c06daf0 2024-01-19T04:55:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f0696ffd640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06980d1e40 0x7f06981564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f38c5514640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f38a406af00 0x7f38a406d3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0079f0 tx=0x7f38bc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f06977fe640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06980cdec0 0x7f06981f35c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f068800d5d0 tx=0x7f068800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7fd919d79640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8f806af20 0x7fd8f806d3e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd904006fd0 tx=0x7fd904008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7fa5e27fc640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d006afa0 0x7fa5d006d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b0530 tx=0x7fa5d8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f49157fa640 1 -- 172.21.15.31:0/2215460726 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490403f070 con 0x7f49181f49e0 2024-01-19T04:55:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f23ceffd640 1 -- 172.21.15.31:0/301204424 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c800b070 con 0x7f23d80ca860 2024-01-19T04:55:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f49157fa640 1 -- 172.21.15.31:0/2215460726 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4904031650 con 0x7f49181f49e0 2024-01-19T04:55:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49180b4620 con 0x7f49181f49e0 2024-01-19T04:55:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f23ceffd640 1 -- 172.21.15.31:0/301204424 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c8041460 con 0x7f23d80ca860 2024-01-19T04:55:28.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f506ffff640 1 -- 172.21.15.31:0/3667861324 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f50881f8ba0 con 0x7f50880b67b0 2024-01-19T04:55:28.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f49157fa640 1 -- 172.21.15.31:0/2215460726 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490403a950 con 0x7f49181f49e0 2024-01-19T04:55:28.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d815d0c0 con 0x7f23d80ca860 2024-01-19T04:55:28.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.838+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49180b4be0 con 0x7f49181f49e0 2024-01-19T04:55:28.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.839+0000 7f23ceffd640 1 -- 172.21.15.31:0/301204424 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8041810 con 0x7f23d80ca860 2024-01-19T04:55:28.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.839+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d80c5ff0 con 0x7f23d80ca860 2024-01-19T04:55:28.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.840+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49180b5230 con 0x7f49181f49e0 2024-01-19T04:55:28.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.840+0000 7f49157fa640 1 -- 172.21.15.31:0/2215460726 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4904043070 con 0x7f49181f49e0 2024-01-19T04:55:28.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.840+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d80c5e50 con 0x7f23d80ca860 2024-01-19T04:55:28.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.840+0000 7f49157fa640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f48fc06aed0 0x7f48fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.841+0000 7f49157fa640 1 -- 172.21.15.31:0/2215460726 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49040ab340 con 0x7f49181f49e0 2024-01-19T04:55:28.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.841+0000 7f4917fff640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f48fc06aed0 0x7f48fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.841+0000 7f23ceffd640 1 -- 172.21.15.31:0/301204424 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f23c8041970 con 0x7f23d80ca860 2024-01-19T04:55:28.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.841+0000 7f4917fff640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f48fc06aed0 0x7f48fc06d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49180ce250 tx=0x7f490c001970 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.841+0000 7f23ceffd640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f23bc06aed0 0x7f23bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.841+0000 7f23ceffd640 1 -- 172.21.15.31:0/301204424 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23c80ab490 con 0x7f23d80ca860 2024-01-19T04:55:28.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.842+0000 7f23dda4f640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f23bc06aed0 0x7f23bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.842+0000 7f23dda4f640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f23bc06aed0 0x7f23bc06d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23d806fc70 tx=0x7f23d0007ef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.842+0000 7fa5cb7fe640 1 -- 172.21.15.31:0/15004949 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa5d40763a0 con 0x7fa5e40cdda0 2024-01-19T04:55:28.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.843+0000 7fd90affd640 1 -- 172.21.15.31:0/4019950649 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd910054990 con 0x7fd91406d5b0 2024-01-19T04:55:28.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.843+0000 7f067ffff640 1 -- 172.21.15.31:0/3679485381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688014620 con 0x7f06980cdec0 2024-01-19T04:55:28.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.843+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069815f680 con 0x7f06980cdec0 2024-01-19T04:55:28.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.843+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06981569e0 con 0x7f06980cdec0 2024-01-19T04:55:28.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.845+0000 7f4b58ff9640 1 -- 172.21.15.31:0/2555989826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4b4c013470 con 0x7f4b5c06daf0 2024-01-19T04:55:28.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.845+0000 7f49157fa640 1 -- 172.21.15.31:0/2215460726 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4904076400 con 0x7f49181f49e0 2024-01-19T04:55:28.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.845+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069815f4e0 con 0x7f06980cdec0 2024-01-19T04:55:28.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.845+0000 7f4b58ff9640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4b4806af20 0x7f4b4806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.846+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4b4806af20 0x7f4b4806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.846+0000 7f4b5b7fe640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4b4806af20 0x7f4b4806d3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b50036000 tx=0x7f4b5003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:28.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.848+0000 7f067ffff640 1 -- 172.21.15.31:0/3679485381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688014780 con 0x7f06980cdec0 2024-01-19T04:55:28.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.848+0000 7f067ffff640 1 -- 172.21.15.31:0/3679485381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688014af0 con 0x7f06980cdec0 2024-01-19T04:55:28.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.849+0000 7f4b58ff9640 1 -- 172.21.15.31:0/2555989826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b4c059060 con 0x7f4b5c06daf0 2024-01-19T04:55:28.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.849+0000 7f38b5ffb640 1 -- 172.21.15.31:0/3193726951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f38b8055300 con 0x7f38c0168810 2024-01-19T04:55:28.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.850+0000 7f23ceffd640 1 -- 172.21.15.31:0/301204424 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f23d80c5e50 con 0x7f23d80ca860 2024-01-19T04:55:28.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.857+0000 7f4b58ff9640 1 -- 172.21.15.31:0/2555989826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4b4c054ca0 con 0x7f4b5c06daf0 2024-01-19T04:55:28.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.859+0000 7f067ffff640 1 -- 172.21.15.31:0/3679485381 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0688014c50 con 0x7f06980cdec0 2024-01-19T04:55:28.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.860+0000 7f067ffff640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f068406aed0 0x7f068406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:28.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.860+0000 7f067ffff640 1 -- 172.21.15.31:0/3679485381 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0688058c50 con 0x7f06980cdec0 2024-01-19T04:55:28.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.865+0000 7f0696ffd640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f068406aed0 0x7f068406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:28.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.867+0000 7f067ffff640 1 -- 172.21.15.31:0/3679485381 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0688055720 con 0x7f06980cdec0 2024-01-19T04:55:28.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:28.881+0000 7f0696ffd640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f068406aed0 0x7f068406d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f068c000c00 tx=0x7f068c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:29.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.242+0000 7f506dffb640 1 -- 172.21.15.31:0/3667861324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50881f8ba0 con 0x7f50880b67b0 2024-01-19T04:55:29.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.243+0000 7f506ffff640 1 -- 172.21.15.31:0/3667861324 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50800775d0 con 0x7f50880b67b0 2024-01-19T04:55:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2024-01-19T04:55:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.246+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f507406af50 msgr2=0x7f507406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.246+0000 7f508df68640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f507406af50 0x7f507406d410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5078006fd0 tx=0x7f5078008040 comp rx=0 tx=0).stop 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.246+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50880b67b0 msgr2=0x7f508815eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.246+0000 7f508df68640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50880b67b0 0x7f508815eb80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5080032000 tx=0x7f5080004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.246+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 shutdown_connections 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.246+0000 7f508df68640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f507406af50 0x7f507406d410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.247+0000 7f508df68640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880cffb0 0x7f50881f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.247+0000 7f508df68640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50880b67b0 0x7f508815eb80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.247+0000 7f508df68640 1 --2- 172.21.15.31:0/3667861324 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50880b48e0 0x7f50881622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.247+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 >> 172.21.15.31:0/3667861324 conn(0x7f50880d3310 msgr2=0x7f50880d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.247+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 shutdown_connections 2024-01-19T04:55:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.247+0000 7f508df68640 1 -- 172.21.15.31:0/3667861324 wait complete. 2024-01-19T04:55:29.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.257+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa5e4153890 con 0x7fa5e40cdda0 2024-01-19T04:55:29.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2024-01-19T04:55:29.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.261+0000 7fa5cb7fe640 1 -- 172.21.15.31:0/15004949 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5d40799c0 con 0x7fa5e40cdda0 2024-01-19T04:55:29.261 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d006afa0 msgr2=0x7fa5d006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d006afa0 0x7fa5d006d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b0530 tx=0x7fa5d8008040 comp rx=0 tx=0).stop 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e40cdda0 msgr2=0x7fa5e41f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e40cdda0 0x7fa5e41f6b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4033000 tx=0x7fa5d40046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 shutdown_connections 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d006afa0 0x7fa5d006d460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e4140e20 0x7fa5e40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e413e0e0 0x7fa5e413c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 --2- 172.21.15.31:0/15004949 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5e40cdda0 0x7fa5e41f6b30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 >> 172.21.15.31:0/15004949 conn(0x7fa5e40b0b60 msgr2=0x7fa5e41404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 shutdown_connections 2024-01-19T04:55:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.263+0000 7fa5e955a640 1 -- 172.21.15.31:0/15004949 wait complete. 2024-01-19T04:55:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.264+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd914074900 con 0x7fd91406d5b0 2024-01-19T04:55:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.265+0000 7fd90affd640 1 -- 172.21.15.31:0/4019950649 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd910015020 con 0x7fd91406d5b0 2024-01-19T04:55:29.265 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2024-01-19T04:55:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.265+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f38a0005810 con 0x7f38c0168810 2024-01-19T04:55:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.266+0000 7f38b5ffb640 1 -- 172.21.15.31:0/3193726951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38b8055300 con 0x7f38c0168810 2024-01-19T04:55:29.266 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.266+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f49180b4d70 con 0x7f49181f49e0 2024-01-19T04:55:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.267+0000 7f49157fa640 1 -- 172.21.15.31:0/2215460726 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4904079a20 con 0x7f49181f49e0 2024-01-19T04:55:29.267 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2024-01-19T04:55:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8f806af20 msgr2=0x7fd8f806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7fd91b57c640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8f806af20 0x7fd8f806d3e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd904006fd0 tx=0x7fd904008040 comp rx=0 tx=0).stop 2024-01-19T04:55:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91406d5b0 msgr2=0x7fd914163790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7fd91b57c640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91406d5b0 0x7fd914163790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b450 tx=0x7fd91000b910 comp rx=0 tx=0).stop 2024-01-19T04:55:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f38a406af00 msgr2=0x7f38a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7f38c7778640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f38a406af00 0x7f38a406d3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0079f0 tx=0x7f38bc009040 comp rx=0 tx=0).stop 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c0168810 msgr2=0x7f38c01635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.268+0000 7f38c7778640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c0168810 0x7f38c01635d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38c00728e0 tx=0x7f38b8007700 comp rx=0 tx=0).stop 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 shutdown_connections 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f38a406af00 0x7f38a406d3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c0168810 0x7f38c01635d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38c0162020 0x7f38c01682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 --2- 172.21.15.31:0/3193726951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38c0089b40 0x7f38c00cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 >> 172.21.15.31:0/3193726951 conn(0x7f38c00b0b60 msgr2=0x7f38c00af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 shutdown_connections 2024-01-19T04:55:29.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8f806af20 0x7fd8f806d3e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd914165390 0x7fd91415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd914146d30 0x7fd9140b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 --2- 172.21.15.31:0/4019950649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91406d5b0 0x7fd914163790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 >> 172.21.15.31:0/4019950649 conn(0x7fd9140b0bd0 msgr2=0x7fd9140714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 shutdown_connections 2024-01-19T04:55:29.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 shutdown_connections 2024-01-19T04:55:29.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7fd91b57c640 1 -- 172.21.15.31:0/4019950649 wait complete. 2024-01-19T04:55:29.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.269+0000 7f38c7778640 1 -- 172.21.15.31:0/3193726951 wait complete. 2024-01-19T04:55:29.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f48fc06aed0 msgr2=0x7f48fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f48fc06aed0 0x7f48fc06d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49180ce250 tx=0x7f490c001970 comp rx=0 tx=0).stop 2024-01-19T04:55:29.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49181f49e0 msgr2=0x7f49180b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49181f49e0 0x7f49180b53e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f490400a120 tx=0x7f4904004730 comp rx=0 tx=0).stop 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 shutdown_connections 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f48fc06aed0 0x7f48fc06d390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49180c3ad0 0x7f49181f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49180b3e90 0x7f49181f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 --2- 172.21.15.31:0/2215460726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49181f49e0 0x7f49180b53e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.270+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 >> 172.21.15.31:0/2215460726 conn(0x7f49180b1a40 msgr2=0x7f4918071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.271+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 shutdown_connections 2024-01-19T04:55:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.271+0000 7f491db68640 1 -- 172.21.15.31:0/2215460726 wait complete. 2024-01-19T04:55:29.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.272+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f23d80c5e50 con 0x7f23d80ca860 2024-01-19T04:55:29.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.273+0000 7f23ceffd640 1 -- 172.21.15.31:0/301204424 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23d80c5e50 con 0x7f23d80ca860 2024-01-19T04:55:29.273 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2024-01-19T04:55:29.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.275+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4b5c15b8a0 con 0x7f4b5c06daf0 2024-01-19T04:55:29.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.276+0000 7f4b58ff9640 1 -- 172.21.15.31:0/2555989826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b4c021090 con 0x7f4b5c06daf0 2024-01-19T04:55:29.276 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2024-01-19T04:55:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f23bc06aed0 msgr2=0x7f23bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f23bc06aed0 0x7f23bc06d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23d806fc70 tx=0x7f23d0007ef0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 msgr2=0x7f23d806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d806d840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23c803a040 tx=0x7f23c80046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 shutdown_connections 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f23bc06aed0 0x7f23bc06d390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d815a3a0 0x7f23d815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23d80cb4c0 0x7f23d81540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 --2- 172.21.15.31:0/301204424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23d80ca860 0x7f23d806d840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 >> 172.21.15.31:0/301204424 conn(0x7f23d80d0000 msgr2=0x7f23d80b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 shutdown_connections 2024-01-19T04:55:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.277+0000 7f23e04b4640 1 -- 172.21.15.31:0/301204424 wait complete. 2024-01-19T04:55:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4b4806af20 msgr2=0x7f4b4806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4b4806af20 0x7f4b4806d3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b50036000 tx=0x7f4b5003d040 comp rx=0 tx=0).stop 2024-01-19T04:55:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b5c06daf0 msgr2=0x7f4b5c14ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b5c06daf0 0x7f4b5c14ae80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00e840 tx=0x7f4b4c00ed00 comp rx=0 tx=0).stop 2024-01-19T04:55:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 shutdown_connections 2024-01-19T04:55:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4b4806af20 0x7f4b4806d3e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c159460 0x7f4b5c152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4b5c070330 0x7f4b5c1ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 --2- 172.21.15.31:0/2555989826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b5c06daf0 0x7f4b5c14ae80 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00e840 tx=0x7f4b4c00ed00 comp rx=0 tx=0).stop 2024-01-19T04:55:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 >> 172.21.15.31:0/2555989826 conn(0x7f4b5c0b0bd0 msgr2=0x7f4b5c149960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-01-19T04:55:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 shutdown_connections 2024-01-19T04:55:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.280+0000 7f4b610d9640 1 -- 172.21.15.31:0/2555989826 wait complete. 2024-01-19T04:55:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.283+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f069815f4e0 con 0x7f06980cdec0 2024-01-19T04:55:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.284+0000 7f067ffff640 1 -- 172.21.15.31:0/3679485381 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0688055720 con 0x7f06980cdec0 2024-01-19T04:55:29.283 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2024-01-19T04:55:29.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f068406aed0 msgr2=0x7f068406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f068406aed0 0x7f068406d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f068c000c00 tx=0x7f068c03d040 comp rx=0 tx=0).stop 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06980cdec0 msgr2=0x7f06981f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06980cdec0 0x7f06981f35c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f068800d5d0 tx=0x7f068800da90 comp rx=0 tx=0).stop 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 shutdown_connections 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f068406aed0 0x7f068406d390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f069815d5c0 0x7f06980cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06980d1e40 0x7f06981564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 --2- 172.21.15.31:0/3679485381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06980cdec0 0x7f06981f35c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.287+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 >> 172.21.15.31:0/3679485381 conn(0x7f069809c600 msgr2=0x7f069806c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.288+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 shutdown_connections 2024-01-19T04:55:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:29.288+0000 7f069db97640 1 -- 172.21.15.31:0/3679485381 wait complete. 2024-01-19T04:55:29.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-01-19T04:55:29.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-01-19T04:55:29.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-01-19T04:55:29.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2024-01-19T04:55:29.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-01-19T04:55:29.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-01-19T04:55:30.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-19T04:55:30.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-19T04:55:30.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-19T04:55:30.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-19T04:55:30.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-19T04:55:30.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-19T04:55:30.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-19T04:55:30.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-19T04:55:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.430+0000 7ff734c3d640 1 Processor -- start 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.430+0000 7ff734c3d640 1 -- start start 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.432+0000 7ff734c3d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff7280eac70 0x7ff728052680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.432+0000 7ff734c3d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff72803b0b0 0x7ff7280483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.432+0000 7ff734c3d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 0x7ff728048900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.432+0000 7ff734c3d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff728052bc0 con 0x7ff72803b0b0 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.432+0000 7ff734c3d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72803a0a0 con 0x7ff7280eac70 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.432+0000 7ff734c3d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72803ac20 con 0x7ff7280eb050 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff72803b0b0 0x7ff7280483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff72803b0b0 0x7ff7280483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43022/0 (socket says 172.21.15.31:43022) 2024-01-19T04:55:30.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 -- 172.21.15.31:0/50303435 learned_addr learned my addr 172.21.15.31:0/50303435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 -- 172.21.15.31:0/50303435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 msgr2=0x7ff728048900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-19T04:55:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 --2- 172.21.15.31:0/50303435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 0x7ff728048900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 -- 172.21.15.31:0/50303435 >> v2:172.21.15.137:3300/0 conn(0x7ff7280eac70 msgr2=0x7ff728052680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-19T04:55:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 --2- 172.21.15.31:0/50303435 >> v2:172.21.15.137:3300/0 conn(0x7ff7280eac70 0x7ff728052680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.433+0000 7ff72dd9b640 1 -- 172.21.15.31:0/50303435 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72803a2f0 con 0x7ff72803b0b0 2024-01-19T04:55:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.435+0000 7ff72dd9b640 1 --2- 172.21.15.31:0/50303435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff72803b0b0 0x7ff7280483c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff730063c30 tx=0x7ff7300683b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d775eab29f329c98 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.435+0000 7ff72d59a640 1 -- 172.21.15.31:0/50303435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300a6d20 con 0x7ff72803b0b0 2024-01-19T04:55:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.435+0000 7ff72d59a640 1 -- 172.21.15.31:0/50303435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7300ae440 con 0x7ff72803b0b0 2024-01-19T04:55:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.436+0000 7ff734c3d640 1 -- 172.21.15.31:0/50303435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff72803b0b0 msgr2=0x7ff7280483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.436+0000 7ff734c3d640 1 --2- 172.21.15.31:0/50303435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff72803b0b0 0x7ff7280483c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff730063c30 tx=0x7ff7300683b0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.436+0000 7ff734c3d640 1 -- 172.21.15.31:0/50303435 shutdown_connections 2024-01-19T04:55:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.436+0000 7ff734c3d640 1 --2- 172.21.15.31:0/50303435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 0x7ff728048900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.436+0000 7ff734c3d640 1 --2- 172.21.15.31:0/50303435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff72803b0b0 0x7ff7280483c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.436+0000 7ff734c3d640 1 --2- 172.21.15.31:0/50303435 >> v2:172.21.15.137:3300/0 conn(0x7ff7280eac70 0x7ff728052680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.436+0000 7ff734c3d640 1 -- 172.21.15.31:0/50303435 >> 172.21.15.31:0/50303435 conn(0x7ff72801e6c0 msgr2=0x7ff7280553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.437+0000 7ff734c3d640 1 -- 172.21.15.31:0/50303435 shutdown_connections 2024-01-19T04:55:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.437+0000 7ff734c3d640 1 -- 172.21.15.31:0/50303435 wait complete. 2024-01-19T04:55:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.439+0000 7ff734c3d640 1 Processor -- start 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.439+0000 7ff734c3d640 1 -- start start 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.440+0000 7ff734c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff72803b0b0 0x7ff728005cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.440+0000 7ff734c3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7280eac70 0x7ff7280c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff734c3d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 0x7ff7280d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff72dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7280eac70 0x7ff7280c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff72dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7280eac70 0x7ff7280c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43028/0 (socket says 172.21.15.31:43028) 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff72e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff72803b0b0 0x7ff728005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff72dd9b640 1 -- 172.21.15.31:0/3028475969 learned_addr learned my addr 172.21.15.31:0/3028475969 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff734c3d640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280c86f0 con 0x7ff7280eac70 2024-01-19T04:55:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff734c3d640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280c72d0 con 0x7ff72803b0b0 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.441+0000 7ff734c3d640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280c6d10 con 0x7ff7280eb050 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.442+0000 7ff72dd9b640 1 -- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 msgr2=0x7ff7280d47a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.442+0000 7ff72dd9b640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 0x7ff7280d47a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.442+0000 7ff72dd9b640 1 -- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff72803b0b0 msgr2=0x7ff728005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.442+0000 7ff72dd9b640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff72803b0b0 0x7ff728005cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.442+0000 7ff72dd9b640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73004eee0 con 0x7ff7280eac70 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.443+0000 7ff72dd9b640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7280eac70 0x7ff7280c6690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff73004ee90 tx=0x7ff7300643a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.443+0000 7ff71effd640 1 -- 172.21.15.31:0/3028475969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300ada10 con 0x7ff7280eac70 2024-01-19T04:55:30.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.443+0000 7ff734c3d640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7280c6ff0 con 0x7ff7280eac70 2024-01-19T04:55:30.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.443+0000 7ff734c3d640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7280c8be0 con 0x7ff7280eac70 2024-01-19T04:55:30.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.447+0000 7ff71effd640 1 -- 172.21.15.31:0/3028475969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7300adb70 con 0x7ff7280eac70 2024-01-19T04:55:30.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.447+0000 7ff71effd640 1 -- 172.21.15.31:0/3028475969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300adee0 con 0x7ff7280eac70 2024-01-19T04:55:30.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.448+0000 7ff71effd640 1 -- 172.21.15.31:0/3028475969 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff7300ba450 con 0x7ff7280eac70 2024-01-19T04:55:30.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.449+0000 7ff734c3d640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff714005290 con 0x7ff7280eac70 2024-01-19T04:55:30.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.451+0000 7ff71effd640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff71806b160 0x7ff71806d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.452+0000 7ff71effd640 1 -- 172.21.15.31:0/3028475969 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff730126300 con 0x7ff7280eac70 2024-01-19T04:55:30.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.459+0000 7ff71effd640 1 -- 172.21.15.31:0/3028475969 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff7300f1120 con 0x7ff7280eac70 2024-01-19T04:55:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.469+0000 7ff72e59c640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff71806b160 0x7ff71806d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.475+0000 7ff72e59c640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff71806b160 0x7ff71806d620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff72000a7d0 tx=0x7ff720008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.483+0000 7fa5f8e31640 1 Processor -- start 2024-01-19T04:55:30.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.483+0000 7fa5f8e31640 1 -- start start 2024-01-19T04:55:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7f038d610640 1 Processor -- start 2024-01-19T04:55:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7f038d610640 1 -- start start 2024-01-19T04:55:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f038d610640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03800df220 0x7f0380038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f038d610640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f038004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.484+0000 7fda1cc09640 1 Processor -- start 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.484+0000 7fda1cc09640 1 -- start start 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fda1cc09640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fda18159460 0x7fda1815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fda1cc09640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda1806dcc0 0x7fda181640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7f2d67446640 1 Processor -- start 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7f2d67446640 1 -- start start 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f2d67446640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d600cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f2d67446640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6015f8d0 con 0x7f2d6015f4f0 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f2d67446640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d600714e0 con 0x7f2d600ce2c0 2024-01-19T04:55:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f2d67446640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d60071e80 con 0x7f2d6006acc0 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f6f4d3f4640 1 Processor -- start 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f6f4d3f4640 1 -- start start 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f8e31640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5f415be00 0x7fa5f4070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f8e31640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa5f40cef80 0x7fa5f40cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f8e31640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f40b21b0 con 0x7fa5f4161e40 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f8e31640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f4071240 con 0x7fa5f415be00 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f8e31640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40b2ee0 con 0x7fa5f40cef80 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f2d9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa5f40cef80 0x7fa5f40cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f2d9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa5f40cef80 0x7fa5f40cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56176/0 (socket says 172.21.15.31:56176) 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.485+0000 7fa5f2d9d640 1 -- 172.21.15.31:0/1422231080 learned_addr learned my addr 172.21.15.31:0/1422231080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fa5f259c640 1 --2- 172.21.15.31:0/1422231080 >> v2:172.21.15.31:3300/0 conn(0x7fa5f415be00 0x7fa5f4070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fa5f2d9d640 1 -- 172.21.15.31:0/1422231080 >> v2:172.21.15.31:3300/0 conn(0x7fa5f415be00 msgr2=0x7fa5f4070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fa5f2d9d640 1 --2- 172.21.15.31:0/1422231080 >> v2:172.21.15.31:3300/0 conn(0x7fa5f415be00 0x7fa5f4070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fa5f2d9d640 1 -- 172.21.15.31:0/1422231080 >> v1:172.21.15.31:6789/0 conn(0x7fa5f4161e40 legacy=0x7fa5f40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fa5f2d9d640 1 -- 172.21.15.31:0/1422231080 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f4165430 con 0x7fa5f40cef80 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f2d649e1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d600cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f2d649e1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d600cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36328/0 (socket says 172.21.15.31:36328) 2024-01-19T04:55:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f2d649e1640 1 -- 172.21.15.31:0/1710361869 learned_addr learned my addr 172.21.15.31:0/1710361869 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f03877fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f038004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f0387fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03800df220 0x7f0380038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f0387fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03800df220 0x7f0380038080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43044/0 (socket says 172.21.15.31:43044) 2024-01-19T04:55:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f4d3f4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f400e30c0 0x7f6f40057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f4d3f4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6f4003e840 0x7f6f40053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f4d3f4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f40054130 0x7f6f400e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f038d610640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03800544c0 con 0x7f0380033ef0 2024-01-19T04:55:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f038d610640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03800541e0 con 0x7f03800df220 2024-01-19T04:55:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f038d610640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0380005f30 con 0x7f0380053c70 2024-01-19T04:55:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f4d3f4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f400061a0 con 0x7f6f400e30c0 2024-01-19T04:55:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f47fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f400e30c0 0x7f6f40057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f47fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f400e30c0 0x7f6f40057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43048/0 (socket says 172.21.15.31:43048) 2024-01-19T04:55:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f47fff640 1 -- 172.21.15.31:0/3159218800 learned_addr learned my addr 172.21.15.31:0/3159218800 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/3159218800 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f400545a0 con 0x7f6f4003e840 2024-01-19T04:55:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/3159218800 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400582d0 con 0x7f6f40054130 2024-01-19T04:55:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7fd412a2f640 1 Processor -- start 2024-01-19T04:55:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7fd412a2f640 1 -- start start 2024-01-19T04:55:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f4cbf3640 1 --2- 172.21.15.31:0/3159218800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f40054130 0x7f6f400e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f6f477fe640 1 --2- 172.21.15.31:0/3159218800 >> v2:172.21.15.137:3300/0 conn(0x7f6f4003e840 0x7f6f40053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f2d57fff640 1 -- 172.21.15.31:0/1710361869 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758380082 0 0) 0x7f2d58001110 con 0x7f2d6015f4f0 2024-01-19T04:55:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f2d57fff640 1 -- 172.21.15.31:0/1710361869 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4c003520 con 0x7f2d6015f4f0 2024-01-19T04:55:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f181a1c9640 1 Processor -- start 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7f181a1c9640 1 -- start start 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fda177fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fda18159460 0x7fda1815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.486+0000 7fda177fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fda18159460 0x7fda1815d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56186/0 (socket says 172.21.15.31:56186) 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7fda177fe640 1 -- 172.21.15.31:0/3144680699 learned_addr learned my addr 172.21.15.31:0/3144680699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda180cd120 con 0x7fda18070330 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda18159840 con 0x7fda1806dcc0 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1815de70 con 0x7fda18159460 2024-01-19T04:55:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7fda17fff640 1 --2- 172.21.15.31:0/3144680699 >> v2:172.21.15.31:3300/0 conn(0x7fda1806dcc0 0x7fda181640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f6f47fff640 1 -- 172.21.15.31:0/3159218800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f40054130 msgr2=0x7f6f400e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:30.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f6f47fff640 1 --2- 172.21.15.31:0/3159218800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f40054130 0x7f6f400e6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.487+0000 7f0387fff640 1 -- 172.21.15.31:0/3785441856 learned_addr learned my addr 172.21.15.31:0/3785441856 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f6f47fff640 1 -- 172.21.15.31:0/3159218800 >> v2:172.21.15.137:3300/0 conn(0x7f6f4003e840 msgr2=0x7f6f40053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:30.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f2d649e1640 1 -- 172.21.15.31:0/1710361869 >> v1:172.21.15.137:6789/0 conn(0x7f2d600ce2c0 legacy=0x7f2d600ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f2d649e1640 1 -- 172.21.15.31:0/1710361869 >> v1:172.21.15.31:6789/0 conn(0x7f2d6015f4f0 legacy=0x7f2d6006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f2d649e1640 1 -- 172.21.15.31:0/1710361869 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d600717c0 con 0x7f2d6006acc0 2024-01-19T04:55:30.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f6f47fff640 1 --2- 172.21.15.31:0/3159218800 >> v2:172.21.15.137:3300/0 conn(0x7f6f4003e840 0x7f6f40053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f6f47fff640 1 -- 172.21.15.31:0/3159218800 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40058570 con 0x7f6f400e30c0 2024-01-19T04:55:30.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fa5f2d9d640 1 --2- 172.21.15.31:0/1422231080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f40cef80 0x7fa5f40cf380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40cfe10 tx=0x7fa5dc03c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131e3170fa0772ed server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fd412a2f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fd412a2f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd40c16abe0 0x7fd40c16b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fd412a2f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c070740 con 0x7fd40c06d4f0 2024-01-19T04:55:30.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fd412a2f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c070560 con 0x7fd40c16abe0 2024-01-19T04:55:30.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fd412a2f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c070d00 con 0x7fd40c072500 2024-01-19T04:55:30.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fda177fe640 1 -- 172.21.15.31:0/3144680699 >> v2:172.21.15.31:3300/0 conn(0x7fda1806dcc0 msgr2=0x7fda181640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fda177fe640 1 --2- 172.21.15.31:0/3144680699 >> v2:172.21.15.31:3300/0 conn(0x7fda1806dcc0 0x7fda181640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fda177fe640 1 -- 172.21.15.31:0/3144680699 >> v1:172.21.15.137:6789/0 conn(0x7fda18070330 legacy=0x7fda1806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fda177fe640 1 -- 172.21.15.31:0/3144680699 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180b3610 con 0x7fda18159460 2024-01-19T04:55:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f0387fff640 1 -- 172.21.15.31:0/3785441856 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 msgr2=0x7f038004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f0387fff640 1 --2- 172.21.15.31:0/3785441856 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f038004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7f0387fff640 1 -- 172.21.15.31:0/3785441856 >> v1:172.21.15.31:6789/0 conn(0x7f0380033ef0 legacy=0x7f0380006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f0387fff640 1 -- 172.21.15.31:0/3785441856 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0380006210 con 0x7f03800df220 2024-01-19T04:55:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.488+0000 7fa5f159a640 1 -- 172.21.15.31:0/1422231080 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc041870 con 0x7fa5f40cef80 2024-01-19T04:55:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fa5f159a640 1 -- 172.21.15.31:0/1422231080 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc0419d0 con 0x7fa5f40cef80 2024-01-19T04:55:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fa5f159a640 1 -- 172.21.15.31:0/1422231080 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc041870 con 0x7fa5f40cef80 2024-01-19T04:55:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f181a1c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18140cdf40 0x7f18140ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f181a1c9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1814166040 con 0x7f1814164e90 2024-01-19T04:55:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f181a1c9640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1814166570 con 0x7f181406acc0 2024-01-19T04:55:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f181a1c9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18140af850 con 0x7f18140cdf40 2024-01-19T04:55:30.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd410fcc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd40c16abe0 0x7fd40c16b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd410fcc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd40c16abe0 0x7fd40c16b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56212/0 (socket says 172.21.15.31:56212) 2024-01-19T04:55:30.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd410fcc640 1 -- 172.21.15.31:0/1283557781 learned_addr learned my addr 172.21.15.31:0/1283557781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40bfff640 1 --2- 172.21.15.31:0/1283557781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40b7fe640 1 --1- 172.21.15.31:0/1283557781 >> v1:172.21.15.137:6789/0 conn(0x7fd40c06d4f0 0x7fd40c16a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:42894/0 (socket says 172.21.15.31:42894) 2024-01-19T04:55:30.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f6f47fff640 1 --2- 172.21.15.31:0/3159218800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f40057d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f400e7cd0 tx=0x7f6f3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39926f3e9373602 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40affd640 1 -- 172.21.15.31:0/1283557781 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558627486 0 0) 0x7fd40c070740 con 0x7fd40c06d4f0 2024-01-19T04:55:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40affd640 1 -- 172.21.15.31:0/1283557781 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f4003520 con 0x7fd40c06d4f0 2024-01-19T04:55:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40bfff640 1 -- 172.21.15.31:0/1283557781 >> v2:172.21.15.137:3300/0 conn(0x7fd40c16abe0 msgr2=0x7fd40c16b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40bfff640 1 --2- 172.21.15.31:0/1283557781 >> v2:172.21.15.137:3300/0 conn(0x7fd40c16abe0 0x7fd40c16b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40bfff640 1 -- 172.21.15.31:0/1283557781 >> v1:172.21.15.137:6789/0 conn(0x7fd40c06d4f0 legacy=0x7fd40c16a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd40bfff640 1 -- 172.21.15.31:0/1283557781 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c070a20 con 0x7fd40c072500 2024-01-19T04:55:30.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fd410fcc640 1 --2- 172.21.15.31:0/1283557781 >> v2:172.21.15.137:3300/0 conn(0x7fd40c16abe0 0x7fd40c16b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7fda177fe640 1 --2- 172.21.15.31:0/3144680699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda18159460 0x7fda1815d930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda1806beb0 tx=0x7fda0c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8903764a60b7cb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f18137fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1814164e90 0x7f181406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59222/0 (socket says 172.21.15.31:59222) 2024-01-19T04:55:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f18137fe640 1 -- 172.21.15.31:0/758814586 learned_addr learned my addr 172.21.15.31:0/758814586 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fda167fc640 1 -- 172.21.15.31:0/3144680699 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c031870 con 0x7fda18159460 2024-01-19T04:55:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fda167fc640 1 -- 172.21.15.31:0/3144680699 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c0319d0 con 0x7fda18159460 2024-01-19T04:55:30.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fda167fc640 1 -- 172.21.15.31:0/3144680699 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c031d60 con 0x7fda18159460 2024-01-19T04:55:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 -- 172.21.15.31:0/1422231080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f40cef80 msgr2=0x7fa5f40cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7f1813fff640 1 --2- 172.21.15.31:0/758814586 >> v2:172.21.15.31:3300/0 conn(0x7f18140cdf40 0x7f18140ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 --2- 172.21.15.31:0/1422231080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f40cef80 0x7fa5f40cf380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40cfe10 tx=0x7fa5dc03c9b0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 -- 172.21.15.31:0/1422231080 shutdown_connections 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 --2- 172.21.15.31:0/1422231080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f40cef80 0x7fa5f40cf380 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 --2- 172.21.15.31:0/1422231080 >> v2:172.21.15.31:3300/0 conn(0x7fa5f415be00 0x7fa5f4070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 -- 172.21.15.31:0/1422231080 >> 172.21.15.31:0/1422231080 conn(0x7fa5f406aa00 msgr2=0x7fa5f406ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 -- 172.21.15.31:0/1422231080 shutdown_connections 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fa5f8e31640 1 -- 172.21.15.31:0/1422231080 wait complete. 2024-01-19T04:55:30.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fd40bfff640 1 --2- 172.21.15.31:0/1283557781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c06cfb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0097a0 tx=0x7fd3fc02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48447c93ef8d0f82 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f2d649e1640 1 --2- 172.21.15.31:0/1710361869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d600cdd80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009e80 tx=0x7f2d5c02bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=205b11dc74776214 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f2d57fff640 1 -- 172.21.15.31:0/1710361869 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c02cae0 con 0x7f2d6006acc0 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f2d57fff640 1 -- 172.21.15.31:0/1710361869 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c02cc40 con 0x7f2d6006acc0 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.489+0000 7f2d57fff640 1 -- 172.21.15.31:0/1710361869 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c0343f0 con 0x7f2d6006acc0 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7f2d67446640 1 -- 172.21.15.31:0/1710361869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 msgr2=0x7f2d600cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7f2d67446640 1 --2- 172.21.15.31:0/1710361869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d600cdd80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009e80 tx=0x7f2d5c02bf40 comp rx=0 tx=0).stop 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7f1813fff640 1 -- 172.21.15.31:0/758814586 >> v1:172.21.15.137:6789/0 conn(0x7f181406acc0 legacy=0x7f18140acd90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7f1813fff640 1 -- 172.21.15.31:0/758814586 >> v1:172.21.15.31:6789/0 conn(0x7f1814164e90 legacy=0x7f181406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7f1813fff640 1 -- 172.21.15.31:0/758814586 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814166290 con 0x7f18140cdf40 2024-01-19T04:55:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f6f46ffd640 1 -- 172.21.15.31:0/3159218800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c030860 con 0x7f6f400e30c0 2024-01-19T04:55:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda18159460 msgr2=0x7fda1815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.490+0000 7fda1cc09640 1 --2- 172.21.15.31:0/3144680699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda18159460 0x7fda1815d930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda1806beb0 tx=0x7fda0c02b910 comp rx=0 tx=0).stop 2024-01-19T04:55:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 shutdown_connections 2024-01-19T04:55:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fda1cc09640 1 --2- 172.21.15.31:0/3144680699 >> v2:172.21.15.31:3300/0 conn(0x7fda1806dcc0 0x7fda181640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fda1cc09640 1 --2- 172.21.15.31:0/3144680699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda18159460 0x7fda1815d930 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 >> 172.21.15.31:0/3144680699 conn(0x7fda180b0bd0 msgr2=0x7fda180b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd40affd640 1 -- 172.21.15.31:0/1283557781 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc030610 con 0x7fd40c072500 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd40affd640 1 -- 172.21.15.31:0/1283557781 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc030770 con 0x7fd40c072500 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd40affd640 1 -- 172.21.15.31:0/1283557781 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc030b00 con 0x7fd40c072500 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f2d67446640 1 -- 172.21.15.31:0/1710361869 shutdown_connections 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f2d67446640 1 --2- 172.21.15.31:0/1710361869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d600cdd80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f2d67446640 1 -- 172.21.15.31:0/1710361869 >> 172.21.15.31:0/1710361869 conn(0x7f2d600b1c50 msgr2=0x7f2d600b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 shutdown_connections 2024-01-19T04:55:30.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fda1cc09640 1 -- 172.21.15.31:0/3144680699 wait complete. 2024-01-19T04:55:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f0387fff640 1 --2- 172.21.15.31:0/3785441856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 0x7f0380038080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f037c009870 tx=0x7f037c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23dc48b0171ab62 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f1813fff640 1 --2- 172.21.15.31:0/758814586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140cdf40 0x7f18140ad4a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f18080097a0 tx=0x7f180802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c70b25d63daccf2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f6f46ffd640 1 -- 172.21.15.31:0/3159218800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c0309c0 con 0x7f6f400e30c0 2024-01-19T04:55:30.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f6f46ffd640 1 -- 172.21.15.31:0/3159218800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c030d50 con 0x7f6f400e30c0 2024-01-19T04:55:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f0386ffd640 1 -- 172.21.15.31:0/3785441856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c031640 con 0x7f03800df220 2024-01-19T04:55:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd412a2f640 1 -- 172.21.15.31:0/1283557781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 msgr2=0x7fd40c06cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd412a2f640 1 --2- 172.21.15.31:0/1283557781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c06cfb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0097a0 tx=0x7fd3fc02b960 comp rx=0 tx=0).stop 2024-01-19T04:55:30.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f2d67446640 1 -- 172.21.15.31:0/1710361869 shutdown_connections 2024-01-19T04:55:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd412a2f640 1 -- 172.21.15.31:0/1283557781 shutdown_connections 2024-01-19T04:55:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd412a2f640 1 --2- 172.21.15.31:0/1283557781 >> v2:172.21.15.137:3300/0 conn(0x7fd40c16abe0 0x7fd40c16b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd412a2f640 1 --2- 172.21.15.31:0/1283557781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c06cfb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7fd412a2f640 1 -- 172.21.15.31:0/1283557781 >> 172.21.15.31:0/1283557781 conn(0x7fd40c0b0b60 msgr2=0x7fd40c0b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7fd412a2f640 1 -- 172.21.15.31:0/1283557781 shutdown_connections 2024-01-19T04:55:30.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7fd412a2f640 1 -- 172.21.15.31:0/1283557781 wait complete. 2024-01-19T04:55:30.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7f0386ffd640 1 -- 172.21.15.31:0/3785441856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c0317a0 con 0x7f03800df220 2024-01-19T04:55:30.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7f0386ffd640 1 -- 172.21.15.31:0/3785441856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c031b30 con 0x7f03800df220 2024-01-19T04:55:30.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7fa5f8e31640 1 Processor -- start 2024-01-19T04:55:30.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7fa5f8e31640 1 -- start start 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/3159218800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 msgr2=0x7f6f40057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.492+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/3159218800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f40057d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f400e7cd0 tx=0x7f6f3c02b9a0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 -- 172.21.15.31:0/3785441856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 msgr2=0x7f0380038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 --2- 172.21.15.31:0/3785441856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 0x7f0380038080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f037c009870 tx=0x7f037c02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 -- 172.21.15.31:0/3785441856 shutdown_connections 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 --2- 172.21.15.31:0/3785441856 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f038004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 --2- 172.21.15.31:0/3785441856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 0x7f0380038080 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 -- 172.21.15.31:0/3785441856 >> 172.21.15.31:0/3785441856 conn(0x7f0380009fb0 msgr2=0x7f038000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 -- 172.21.15.31:0/3785441856 shutdown_connections 2024-01-19T04:55:30.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f038d610640 1 -- 172.21.15.31:0/3785441856 wait complete. 2024-01-19T04:55:30.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7fda1cc09640 1 Processor -- start 2024-01-19T04:55:30.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7fda1cc09640 1 -- start start 2024-01-19T04:55:30.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7fd412a2f640 1 Processor -- start 2024-01-19T04:55:30.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.491+0000 7f2d67446640 1 -- 172.21.15.31:0/1710361869 wait complete. 2024-01-19T04:55:30.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f2d67446640 1 Processor -- start 2024-01-19T04:55:30.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/3159218800 shutdown_connections 2024-01-19T04:55:30.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/3159218800 >> v2:172.21.15.137:3300/0 conn(0x7f6f4003e840 0x7f6f40053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/3159218800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f40054130 0x7f6f400e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/3159218800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f40057d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/3159218800 >> 172.21.15.31:0/3159218800 conn(0x7f6f400559e0 msgr2=0x7f6f40055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7f2d67446640 1 -- start start 2024-01-19T04:55:30.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7fa5f8e31640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5f40cef80 0x7fa5f414e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.493+0000 7fd412a2f640 1 -- start start 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f8e31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f415be00 0x7fa5f4149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f8e31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f4161e40 0x7fa5f4150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f259c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5f40cef80 0x7fa5f414e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f259c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5f40cef80 0x7fa5f414e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36358/0 (socket says 172.21.15.31:36358) 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f415be00 0x7fa5f4149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f415be00 0x7fa5f4149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56226/0 (socket says 172.21.15.31:56226) 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f259c640 1 -- 172.21.15.31:0/2421355531 learned_addr learned my addr 172.21.15.31:0/2421355531 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f8e31640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f414dfc0 con 0x7fa5f4161e40 2024-01-19T04:55:30.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f8e31640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f41645d0 con 0x7fa5f415be00 2024-01-19T04:55:30.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fda1cc09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda1806dcc0 0x7fda1815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f8e31640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f414d690 con 0x7fa5f40cef80 2024-01-19T04:55:30.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fd412a2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40c06d4f0 0x7fd40c0cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fda1cc09640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda18070330 0x7fda181465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fd412a2f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fd412a2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd40c16abe0 0x7fd40c15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fd412a2f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c156710 con 0x7fd40c16abe0 2024-01-19T04:55:30.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fd412a2f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0cc6f0 con 0x7fd40c06d4f0 2024-01-19T04:55:30.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fd412a2f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0ce070 con 0x7fd40c072500 2024-01-19T04:55:30.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fda1cc09640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda18159460 0x7fda18151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.494+0000 7fa5f2d9d640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f4161e40 0x7fa5f4150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c0cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36360/0 (socket says 172.21.15.31:36360) 2024-01-19T04:55:30.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 -- 172.21.15.31:0/337662293 learned_addr learned my addr 172.21.15.31:0/337662293 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda1cc09640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda18153250 con 0x7fda18159460 2024-01-19T04:55:30.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda1cc09640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda18152640 con 0x7fda1806dcc0 2024-01-19T04:55:30.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda1cc09640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda18152920 con 0x7fda18070330 2024-01-19T04:55:30.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d67446640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d6015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d67446640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d600ce2c0 0x7f2d60161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40bfff640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40c06d4f0 0x7fd40c0cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/3159218800 shutdown_connections 2024-01-19T04:55:30.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/3159218800 wait complete. 2024-01-19T04:55:30.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 -- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40c06d4f0 msgr2=0x7fd40c0cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40c06d4f0 0x7fd40c0cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 -- 172.21.15.31:0/337662293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd40c16abe0 msgr2=0x7fd40c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd40c16abe0 0x7fd40c15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 -- 172.21.15.31:0/337662293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009b30 con 0x7fd40c072500 2024-01-19T04:55:30.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d651e2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d6015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d651e2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d6015d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36370/0 (socket says 172.21.15.31:36370) 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d651e2640 1 -- 172.21.15.31:0/3987254370 learned_addr learned my addr 172.21.15.31:0/3987254370 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d649e1640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d600ce2c0 0x7f2d60161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d67446640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d6015f4f0 0x7f2d600b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d600c81f0 con 0x7f2d6015f4f0 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d60162570 con 0x7f2d600ce2c0 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d60162850 con 0x7f2d6006acc0 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7f038d610640 1 Processor -- start 2024-01-19T04:55:30.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fd40b7fe640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c0cdb30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c500 tx=0x7fd40000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fa5f2d9d640 1 -- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5f40cef80 msgr2=0x7fa5f414e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fa5f2d9d640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5f40cef80 0x7fa5f414e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fa5f2d9d640 1 -- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f415be00 msgr2=0x7fa5f4149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fa5f2d9d640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f415be00 0x7fa5f4149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fa5f2d9d640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0094d0 con 0x7fa5f4161e40 2024-01-19T04:55:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7f2d659e3640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d6015f4f0 0x7f2d600b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7f038d610640 1 -- start start 2024-01-19T04:55:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fd408ff9640 1 -- 172.21.15.31:0/337662293 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400007500 con 0x7fd40c072500 2024-01-19T04:55:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fd408ff9640 1 -- 172.21.15.31:0/337662293 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd400007660 con 0x7fd40c072500 2024-01-19T04:55:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fd408ff9640 1 -- 172.21.15.31:0/337662293 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400004a20 con 0x7fd40c072500 2024-01-19T04:55:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3fc009400 con 0x7fd40c072500 2024-01-19T04:55:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c0cc410 con 0x7fd40c072500 2024-01-19T04:55:30.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7f2d659e3640 1 -- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 msgr2=0x7f2d6015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7f2d659e3640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d6015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7f2d659e3640 1 -- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d600ce2c0 msgr2=0x7f2d60161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7f2d659e3640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d600ce2c0 0x7f2d60161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7f2d659e3640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c009b30 con 0x7f2d6015f4f0 2024-01-19T04:55:30.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fa5f2d9d640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f4161e40 0x7fa5f4150670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc044000 tx=0x7fa5dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda177fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda1806dcc0 0x7fda1815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda18070330 0x7fda181465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda18070330 0x7fda181465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36366/0 (socket says 172.21.15.31:36366) 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda16ffd640 1 -- 172.21.15.31:0/980839022 learned_addr learned my addr 172.21.15.31:0/980839022 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.495+0000 7fda17fff640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda18159460 0x7fda18151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fda16ffd640 1 -- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda1806dcc0 msgr2=0x7fda1815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fda16ffd640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda1806dcc0 0x7fda1815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fda16ffd640 1 -- 172.21.15.31:0/980839022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda18159460 msgr2=0x7fda18151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fda16ffd640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda18159460 0x7fda18151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fda16ffd640 1 -- 172.21.15.31:0/980839022 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c0094d0 con 0x7fda18070330 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fda16ffd640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda18070330 0x7fda181465c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c430 tx=0x7fda0800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.496+0000 7fd9fffff640 1 -- 172.21.15.31:0/980839022 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08011790 con 0x7fda18070330 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7fd9fffff640 1 -- 172.21.15.31:0/980839022 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda080118f0 con 0x7fda18070330 2024-01-19T04:55:30.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7fd9fffff640 1 -- 172.21.15.31:0/980839022 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08011790 con 0x7fda18070330 2024-01-19T04:55:30.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f2d659e3640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d6015f4f0 0x7f2d600b4f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000c3c0 tx=0x7f2d5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f038d610640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0380033ef0 0x7f038004b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f6f4d3f4640 1 Processor -- start 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f038d610640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f03800ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f038d610640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 0x7f038004d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f0387fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0380033ef0 0x7f038004b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f03877fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f03800ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f038d610640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038004eb60 con 0x7f03800df220 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f038d610640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03800ec460 con 0x7f0380033ef0 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f0387fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0380033ef0 0x7f038004b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56280/0 (socket says 172.21.15.31:56280) 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f0387fff640 1 -- 172.21.15.31:0/2118768805 learned_addr learned my addr 172.21.15.31:0/2118768805 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c074900 con 0x7fd40c072500 2024-01-19T04:55:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.497+0000 7f6f4d3f4640 1 -- start start 2024-01-19T04:55:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f038d610640 1 -- 172.21.15.31:0/2118768805 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038004e2c0 con 0x7f0380053c70 2024-01-19T04:55:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f038ce0f640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 0x7f038004d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda18152c00 con 0x7fda18070330 2024-01-19T04:55:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda18146b00 con 0x7fda18070330 2024-01-19T04:55:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f03877fe640 1 -- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0380033ef0 msgr2=0x7f038004b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f03877fe640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0380033ef0 0x7f038004b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f03877fe640 1 -- 172.21.15.31:0/2118768805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 msgr2=0x7f038004d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f03877fe640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 0x7f038004d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f03877fe640 1 -- 172.21.15.31:0/2118768805 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c0094d0 con 0x7f0380053c70 2024-01-19T04:55:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f0387fff640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0380033ef0 0x7f038004b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7fd408ff9640 1 -- 172.21.15.31:0/337662293 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd400004b80 con 0x7fd40c072500 2024-01-19T04:55:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.498+0000 7f03877fe640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f03800ceca0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f037400e8a0 tx=0x7f037400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f2d55ffb640 1 -- 172.21.15.31:0/3987254370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500052d0 con 0x7f2d6015f4f0 2024-01-19T04:55:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f4d3f4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f4003e840 0x7f6f40051ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda18153400 con 0x7fda18070330 2024-01-19T04:55:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fd9fffff640 1 -- 172.21.15.31:0/980839022 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fda08011aa0 con 0x7fda18070330 2024-01-19T04:55:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fd408ff9640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd3f806af20 0x7fd3f806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fd408ff9640 1 -- 172.21.15.31:0/337662293 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd400088c40 con 0x7fd40c072500 2024-01-19T04:55:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fa5eaffd640 1 -- 172.21.15.31:0/2421355531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc041870 con 0x7fa5f4161e40 2024-01-19T04:55:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fa5f8e31640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f414d970 con 0x7fa5f4161e40 2024-01-19T04:55:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fa5f8e31640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f414e990 con 0x7fa5f4161e40 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d60162b30 con 0x7f2d6015f4f0 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f2d55ffb640 1 -- 172.21.15.31:0/3987254370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d50005430 con 0x7f2d6015f4f0 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f2d55ffb640 1 -- 172.21.15.31:0/3987254370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50007260 con 0x7f2d6015f4f0 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f4d3f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40054130 0x7f6f4004ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f47fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f4003e840 0x7f6f40051ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f47fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f4003e840 0x7f6f40051ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36392/0 (socket says 172.21.15.31:36392) 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f47fff640 1 -- 172.21.15.31:0/2129853392 learned_addr learned my addr 172.21.15.31:0/2129853392 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f4004ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f477fe640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40054130 0x7f6f4004ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40051630 con 0x7f6f400e30c0 2024-01-19T04:55:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40050a20 con 0x7f6f40054130 2024-01-19T04:55:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40050d00 con 0x7f6f4003e840 2024-01-19T04:55:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.499+0000 7fd40bfff640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd3f806af20 0x7fd3f806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7f18127fc640 1 -- 172.21.15.31:0/758814586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808031830 con 0x7f18140cdf40 2024-01-19T04:55:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7f18127fc640 1 -- 172.21.15.31:0/758814586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1808031990 con 0x7f18140cdf40 2024-01-19T04:55:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7f18127fc640 1 -- 172.21.15.31:0/758814586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808031d20 con 0x7f18140cdf40 2024-01-19T04:55:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7fd9fffff640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fda0406af60 0x7fda0406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7fd9fffff640 1 -- 172.21.15.31:0/980839022 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda08099e60 con 0x7fda18070330 2024-01-19T04:55:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7fa5eaffd640 1 -- 172.21.15.31:0/2421355531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc04ae70 con 0x7fa5f4161e40 2024-01-19T04:55:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7f6f4cbf3640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f4004ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc005290 con 0x7fa5f4161e40 2024-01-19T04:55:30.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.500+0000 7fda177fe640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fda0406af60 0x7fda0406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7fa5eaffd640 1 -- 172.21.15.31:0/2421355531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc054640 con 0x7fa5f4161e40 2024-01-19T04:55:30.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f181a1c9640 1 -- 172.21.15.31:0/758814586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140cdf40 msgr2=0x7f18140ad4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f181a1c9640 1 --2- 172.21.15.31:0/758814586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140cdf40 0x7f18140ad4a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f18080097a0 tx=0x7f180802b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f181a1c9640 1 -- 172.21.15.31:0/758814586 shutdown_connections 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f181a1c9640 1 --2- 172.21.15.31:0/758814586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140cdf40 0x7f18140ad4a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f181a1c9640 1 -- 172.21.15.31:0/758814586 >> 172.21.15.31:0/758814586 conn(0x7f18140cff20 msgr2=0x7f18140d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f6f4cbf3640 1 -- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f4003e840 msgr2=0x7f6f40051ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f6f4cbf3640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f4003e840 0x7f6f40051ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f6f4cbf3640 1 -- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40054130 msgr2=0x7f6f4004ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f6f4cbf3640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40054130 0x7f6f4004ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f6f4cbf3640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0094d0 con 0x7f6f400e30c0 2024-01-19T04:55:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f6f4cbf3640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f4004ffa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f4804fbe0 tx=0x7f6f48068110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7fd40bfff640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd3f806af20 0x7fd3f806d3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc036000 tx=0x7fd3fc03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d600cd3f0 con 0x7f2d6015f4f0 2024-01-19T04:55:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.501+0000 7f181a1c9640 1 -- 172.21.15.31:0/758814586 shutdown_connections 2024-01-19T04:55:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.502+0000 7f181a1c9640 1 -- 172.21.15.31:0/758814586 wait complete. 2024-01-19T04:55:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.502+0000 7f2d55ffb640 1 -- 172.21.15.31:0/3987254370 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2d500073c0 con 0x7f2d6015f4f0 2024-01-19T04:55:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.502+0000 7fda177fe640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fda0406af60 0x7fda0406d420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda0c036000 tx=0x7fda0c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.502+0000 7fa5eaffd640 1 -- 172.21.15.31:0/2421355531 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa5dc0547a0 con 0x7fa5f4161e40 2024-01-19T04:55:30.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.502+0000 7f2d55ffb640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d4406af00 0x7f2d4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.503+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d60163050 con 0x7f2d6015f4f0 2024-01-19T04:55:30.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.503+0000 7f2d651e2640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d4406af00 0x7f2d4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.503+0000 7fa5eaffd640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d406af20 0x7fa5d406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.503+0000 7f2d651e2640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d4406af00 0x7f2d4406d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001370 tx=0x7f2d58009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.503+0000 7fa5eaffd640 1 -- 172.21.15.31:0/2421355531 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5dc04f070 con 0x7fa5f4161e40 2024-01-19T04:55:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.503+0000 7fa5f259c640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d406af20 0x7fa5d406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.503+0000 7f181a1c9640 1 Processor -- start 2024-01-19T04:55:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f181a1c9640 1 -- start start 2024-01-19T04:55:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f0384ff9640 1 -- 172.21.15.31:0/2118768805 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374009e60 con 0x7f0380053c70 2024-01-19T04:55:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f038d610640 1 -- 172.21.15.31:0/2118768805 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038004e5a0 con 0x7f0380053c70 2024-01-19T04:55:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f038d610640 1 -- 172.21.15.31:0/2118768805 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03800cf1e0 con 0x7f0380053c70 2024-01-19T04:55:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7fa5f259c640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d406af20 0x7fa5d406d3e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec006fd0 tx=0x7fa5ec008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7fd408ff9640 1 -- 172.21.15.31:0/337662293 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd400053b80 con 0x7fd40c072500 2024-01-19T04:55:30.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f181a1c9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f181406acc0 0x7f18140b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f181a1c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18140cdf40 0x7f181415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f181a1c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1814164e90 0x7f1814161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.504+0000 7f181a1c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c54c0 con 0x7f1814164e90 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f181a1c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140b54c0 con 0x7f18140cdf40 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f181a1c9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140b48b0 con 0x7f181406acc0 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f181406acc0 0x7f18140b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f181406acc0 0x7f18140b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36404/0 (socket says 172.21.15.31:36404) 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 -- 172.21.15.31:0/1971285972 learned_addr learned my addr 172.21.15.31:0/1971285972 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f1812ffd640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18140cdf40 0x7f181415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f1813fff640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1814164e90 0x7f1814161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 -- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18140cdf40 msgr2=0x7f181415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0368005290 con 0x7f0380053c70 2024-01-19T04:55:30.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18140cdf40 0x7f181415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 -- 172.21.15.31:0/1971285972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1814164e90 msgr2=0x7f1814161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1814164e90 0x7f1814161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 -- 172.21.15.31:0/1971285972 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1808009400 con 0x7f181406acc0 2024-01-19T04:55:30.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.505+0000 7f18137fe640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f181406acc0 0x7f18140b42d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f180000c4b0 tx=0x7f180000c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.507+0000 7f2d55ffb640 1 -- 172.21.15.31:0/3987254370 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d50057220 con 0x7f2d6015f4f0 2024-01-19T04:55:30.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.507+0000 7fa5eaffd640 1 -- 172.21.15.31:0/2421355531 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa5dc089010 con 0x7fa5f4161e40 2024-01-19T04:55:30.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.507+0000 7f6f44ff9640 1 -- 172.21.15.31:0/2129853392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48093440 con 0x7f6f400e30c0 2024-01-19T04:55:30.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.507+0000 7f6f44ff9640 1 -- 172.21.15.31:0/2129853392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f480935c0 con 0x7f6f400e30c0 2024-01-19T04:55:30.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.507+0000 7f6f44ff9640 1 -- 172.21.15.31:0/2129853392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f480939b0 con 0x7f6f400e30c0 2024-01-19T04:55:30.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.507+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f40050fe0 con 0x7f6f400e30c0 2024-01-19T04:55:30.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.508+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f40052000 con 0x7f6f400e30c0 2024-01-19T04:55:30.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.508+0000 7f0384ff9640 1 -- 172.21.15.31:0/2118768805 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0374005990 con 0x7f0380053c70 2024-01-19T04:55:30.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.509+0000 7f0384ff9640 1 -- 172.21.15.31:0/2118768805 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374005d00 con 0x7f0380053c70 2024-01-19T04:55:30.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.508+0000 7fd9fffff640 1 -- 172.21.15.31:0/980839022 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fda18153400 con 0x7fda18070330 2024-01-19T04:55:30.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.508+0000 7f17fffff640 1 -- 172.21.15.31:0/1971285972 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800005ce0 con 0x7f181406acc0 2024-01-19T04:55:30.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.508+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18140b4b90 con 0x7f181406acc0 2024-01-19T04:55:30.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.509+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18140b5150 con 0x7f181406acc0 2024-01-19T04:55:30.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.509+0000 7f17fffff640 1 -- 172.21.15.31:0/1971285972 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1800005e40 con 0x7f181406acc0 2024-01-19T04:55:30.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.509+0000 7f17fffff640 1 -- 172.21.15.31:0/1971285972 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800007270 con 0x7f181406acc0 2024-01-19T04:55:30.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.509+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f400517a0 con 0x7f6f400e30c0 2024-01-19T04:55:30.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.510+0000 7f0384ff9640 1 -- 172.21.15.31:0/2118768805 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f037400c940 con 0x7f0380053c70 2024-01-19T04:55:30.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.511+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f4005290 con 0x7f181406acc0 2024-01-19T04:55:30.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.511+0000 7f0384ff9640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f036c06aed0 0x7f036c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.511+0000 7f0384ff9640 1 -- 172.21.15.31:0/2118768805 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0374089d60 con 0x7f0380053c70 2024-01-19T04:55:30.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.511+0000 7f0387fff640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f036c06aed0 0x7f036c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.512+0000 7f0387fff640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f036c06aed0 0x7f036c06d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f037c0090b0 tx=0x7f037c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.513+0000 7f0384ff9640 1 -- 172.21.15.31:0/2118768805 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0374054cc0 con 0x7f0380053c70 2024-01-19T04:55:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.513+0000 7f2d55ffb640 1 -- 172.21.15.31:0/3987254370 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2d50053560 con 0x7f2d6015f4f0 2024-01-19T04:55:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.514+0000 7f17fffff640 1 -- 172.21.15.31:0/1971285972 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f18000073d0 con 0x7f181406acc0 2024-01-19T04:55:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.515+0000 7f17fffff640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f17f806aed0 0x7f17f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.515+0000 7f17fffff640 1 -- 172.21.15.31:0/1971285972 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1800088f40 con 0x7f181406acc0 2024-01-19T04:55:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.517+0000 7f1812ffd640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f17f806aed0 0x7f17f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.516+0000 7f6f44ff9640 1 -- 172.21.15.31:0/2129853392 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6f48093b10 con 0x7f6f400e30c0 2024-01-19T04:55:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.517+0000 7f6f44ff9640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f2c06af20 0x7f6f2c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.517+0000 7f6f47fff640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f2c06af20 0x7f6f2c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.518+0000 7f6f44ff9640 1 -- 172.21.15.31:0/2129853392 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f480d26f0 con 0x7f6f400e30c0 2024-01-19T04:55:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.518+0000 7f1812ffd640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f17f806aed0 0x7f17f806d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f180c000b60 tx=0x7f180c00d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.518+0000 7f6f47fff640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f2c06af20 0x7f6f2c06d3e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009280 tx=0x7f6f3c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.519+0000 7f17fffff640 1 -- 172.21.15.31:0/1971285972 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1800053f30 con 0x7f181406acc0 2024-01-19T04:55:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.524+0000 7f6f44ff9640 1 -- 172.21.15.31:0/2129853392 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6f480cf1c0 con 0x7f6f400e30c0 2024-01-19T04:55:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.868+0000 7ff734c3d640 1 -- 172.21.15.31:0/3028475969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff714005540 con 0x7ff7280eac70 2024-01-19T04:55:30.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.869+0000 7ff71effd640 1 -- 172.21.15.31:0/3028475969 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7300be020 con 0x7ff7280eac70 2024-01-19T04:55:30.868 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.872+0000 7ff71cff9640 1 -- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff71806b160 msgr2=0x7ff71806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.872+0000 7ff71cff9640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff71806b160 0x7ff71806d620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff72000a7d0 tx=0x7ff720008040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.872+0000 7ff71cff9640 1 -- 172.21.15.31:0/3028475969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7280eac70 msgr2=0x7ff7280c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.872+0000 7ff71cff9640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7280eac70 0x7ff7280c6690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff73004ee90 tx=0x7ff7300643a0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 -- 172.21.15.31:0/3028475969 shutdown_connections 2024-01-19T04:55:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff71806b160 0x7ff71806d620 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7280eb050 0x7ff7280d47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7280eac70 0x7ff7280c6690 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 --2- 172.21.15.31:0/3028475969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff72803b0b0 0x7ff728005cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 -- 172.21.15.31:0/3028475969 >> 172.21.15.31:0/3028475969 conn(0x7ff72801e6c0 msgr2=0x7ff728006810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 -- 172.21.15.31:0/3028475969 shutdown_connections 2024-01-19T04:55:30.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.873+0000 7ff71cff9640 1 -- 172.21.15.31:0/3028475969 wait complete. 2024-01-19T04:55:30.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-01-19T04:55:30.889 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.904+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2d60074900 con 0x7f2d6015f4f0 2024-01-19T04:55:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.905+0000 7f2d55ffb640 1 -- 172.21.15.31:0/3987254370 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d50056b80 con 0x7f2d6015f4f0 2024-01-19T04:55:30.904 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d4406af00 msgr2=0x7f2d4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d4406af00 0x7f2d4406d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001370 tx=0x7f2d58009040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d6015f4f0 msgr2=0x7f2d600b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d6015f4f0 0x7f2d600b4f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000c3c0 tx=0x7f2d5000c880 comp rx=0 tx=0).stop 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 shutdown_connections 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d4406af00 0x7f2d4406d3c0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001370 tx=0x7f2d58009040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d6015f4f0 0x7f2d600b4f20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d600ce2c0 0x7f2d60161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 --2- 172.21.15.31:0/3987254370 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d6006acc0 0x7f2d6015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 >> 172.21.15.31:0/3987254370 conn(0x7f2d600b1c50 msgr2=0x7f2d6006e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 shutdown_connections 2024-01-19T04:55:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.907+0000 7f2d67446640 1 -- 172.21.15.31:0/3987254370 wait complete. 2024-01-19T04:55:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.917+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd40c156880 con 0x7fd40c072500 2024-01-19T04:55:30.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-01-19T04:55:30.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.918+0000 7fd408ff9640 1 -- 172.21.15.31:0/337662293 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd400018070 con 0x7fd40c072500 2024-01-19T04:55:30.918 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-01-19T04:55:30.918 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd3f806af20 msgr2=0x7fd3f806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd3f806af20 0x7fd3f806d3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc036000 tx=0x7fd3fc03b040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 msgr2=0x7fd40c0cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c0cdb30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c500 tx=0x7fd40000c9c0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 shutdown_connections 2024-01-19T04:55:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd3f806af20 0x7fd3f806d3e0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc036000 tx=0x7fd3fc03b040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd40c16abe0 0x7fd40c15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd40c072500 0x7fd40c0cdb30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 --2- 172.21.15.31:0/337662293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40c06d4f0 0x7fd40c0cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.920+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 >> 172.21.15.31:0/337662293 conn(0x7fd40c0b0b60 msgr2=0x7fd40c15bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.921+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 shutdown_connections 2024-01-19T04:55:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.921+0000 7fd412a2f640 1 -- 172.21.15.31:0/337662293 wait complete. 2024-01-19T04:55:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.923+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa5cc005810 con 0x7fa5f4161e40 2024-01-19T04:55:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.923+0000 7fa5eaffd640 1 -- 172.21.15.31:0/2421355531 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5dc08c630 con 0x7fa5f4161e40 2024-01-19T04:55:30.923 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:30.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.926+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d406af20 msgr2=0x7fa5d406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.926+0000 7fa5e8ff9640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d406af20 0x7fa5d406d3e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec006fd0 tx=0x7fa5ec008040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.926+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f4161e40 msgr2=0x7fa5f4150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.926+0000 7fa5e8ff9640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f4161e40 0x7fa5f4150670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc044000 tx=0x7fa5dc004730 comp rx=0 tx=0).stop 2024-01-19T04:55:30.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.926+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 shutdown_connections 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fa5e8ff9640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa5d406af20 0x7fa5d406d3e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fa5e8ff9640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5f4161e40 0x7fa5f4150670 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fa5e8ff9640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f415be00 0x7fa5f4149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fa5e8ff9640 1 --2- 172.21.15.31:0/2421355531 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5f40cef80 0x7fa5f414e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 >> 172.21.15.31:0/2421355531 conn(0x7fa5f406aa00 msgr2=0x7fa5f40b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 shutdown_connections 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fa5e8ff9640 1 -- 172.21.15.31:0/2421355531 wait complete. 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fda18074900 con 0x7fda18070330 2024-01-19T04:55:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.927+0000 7fd9fffff640 1 -- 172.21.15.31:0/980839022 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda18153400 con 0x7fda18070330 2024-01-19T04:55:30.928 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.929+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0368005810 con 0x7f0380053c70 2024-01-19T04:55:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7f0384ff9640 1 -- 172.21.15.31:0/2118768805 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0374021090 con 0x7f0380053c70 2024-01-19T04:55:30.929 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fda0406af60 msgr2=0x7fda0406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fda0406af60 0x7fda0406d420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda0c036000 tx=0x7fda0c03c040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda18070330 msgr2=0x7fda181465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda18070330 0x7fda181465c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c430 tx=0x7fda0800c8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 shutdown_connections 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fda0406af60 0x7fda0406d420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda18159460 0x7fda18151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda18070330 0x7fda181465c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 --2- 172.21.15.31:0/980839022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda1806dcc0 0x7fda1815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.930+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 >> 172.21.15.31:0/980839022 conn(0x7fda180b0bd0 msgr2=0x7fda180b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.931+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 shutdown_connections 2024-01-19T04:55:30.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.931+0000 7fda1cc09640 1 -- 172.21.15.31:0/980839022 wait complete. 2024-01-19T04:55:30.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.931+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f17f4005810 con 0x7f181406acc0 2024-01-19T04:55:30.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.932+0000 7f17fffff640 1 -- 172.21.15.31:0/1971285972 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1800017090 con 0x7f181406acc0 2024-01-19T04:55:30.932 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f036c06aed0 msgr2=0x7f036c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f036c06aed0 0x7f036c06d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f037c0090b0 tx=0x7f037c03c040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 msgr2=0x7f03800ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f03800ceca0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f037400e8a0 tx=0x7f037400ed60 comp rx=0 tx=0).stop 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 shutdown_connections 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f036c06aed0 0x7f036c06d390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03800df220 0x7f038004d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0380053c70 0x7f03800ceca0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 --2- 172.21.15.31:0/2118768805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0380033ef0 0x7f038004b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 >> 172.21.15.31:0/2118768805 conn(0x7f0380009fb0 msgr2=0x7f038000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 shutdown_connections 2024-01-19T04:55:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.933+0000 7f03727fc640 1 -- 172.21.15.31:0/2118768805 wait complete. 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f17f806aed0 msgr2=0x7f17f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f17f806aed0 0x7f17f806d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f180c000b60 tx=0x7f180c00d040 comp rx=0 tx=0).stop 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f181406acc0 msgr2=0x7f18140b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f181406acc0 0x7f18140b42d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f180000c4b0 tx=0x7f180000c970 comp rx=0 tx=0).stop 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 shutdown_connections 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f17f806aed0 0x7f17f806d390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1814164e90 0x7f1814161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18140cdf40 0x7f181415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 --2- 172.21.15.31:0/1971285972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f181406acc0 0x7f18140b42d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 >> 172.21.15.31:0/1971285972 conn(0x7f18140cff20 msgr2=0x7f18140c2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 shutdown_connections 2024-01-19T04:55:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.934+0000 7f181a1c9640 1 -- 172.21.15.31:0/1971285972 wait complete. 2024-01-19T04:55:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.935+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6f400057e0 con 0x7f6f400e30c0 2024-01-19T04:55:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.936+0000 7f6f44ff9640 1 -- 172.21.15.31:0/2129853392 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f480cf1c0 con 0x7f6f400e30c0 2024-01-19T04:55:30.937 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-01-19T04:55:30.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-01-19T04:55:30.938 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f2c06af20 msgr2=0x7f6f2c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f2c06af20 0x7f6f2c06d3e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009280 tx=0x7f6f3c008030 comp rx=0 tx=0).stop 2024-01-19T04:55:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 msgr2=0x7f6f4004ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f4004ffa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f4804fbe0 tx=0x7f6f48068110 comp rx=0 tx=0).stop 2024-01-19T04:55:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 shutdown_connections 2024-01-19T04:55:30.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f2c06af20 0x7f6f2c06d3e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f400e30c0 0x7f6f4004ffa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40054130 0x7f6f4004ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 --2- 172.21.15.31:0/2129853392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6f4003e840 0x7f6f40051ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:30.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 >> 172.21.15.31:0/2129853392 conn(0x7f6f400559e0 msgr2=0x7f6f40039f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:30.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 shutdown_connections 2024-01-19T04:55:30.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:30.938+0000 7f6f4d3f4640 1 -- 172.21.15.31:0/2129853392 wait complete. 2024-01-19T04:55:30.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-01-19T04:55:30.947 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-01-19T04:55:30.950 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-01-19T04:55:30.950 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-01-19T04:55:30.951 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-01-19T04:55:30.952 DEBUG:teuthology.parallel:result is None 2024-01-19T04:55:30.952 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-19T04:55:30.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:55:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.093+0000 7f7e16a96640 1 Processor -- start 2024-01-19T04:55:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.093+0000 7f7e16a96640 1 -- start start 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.094+0000 7f7e16a96640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7e1006e8d0 0x7f7e100d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.094+0000 7f7e16a96640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e1006ecb0 0x7f7e100cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.094+0000 7f7e16a96640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e100d0040 con 0x7f7e100cee00 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.094+0000 7f7e16a96640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e100cd500 con 0x7f7e1006e8d0 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.094+0000 7f7e16a96640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e100cb800 con 0x7f7e1006ecb0 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.094+0000 7f7e0ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7e100cee00 0x7f7e1015c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59238/0 (socket says 172.21.15.31:59238) 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.094+0000 7f7e0ffff640 1 -- 172.21.15.31:0/3198322965 learned_addr learned my addr 172.21.15.31:0/3198322965 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e14832640 1 --2- 172.21.15.31:0/3198322965 >> v2:172.21.15.137:3300/0 conn(0x7f7e1006e8d0 0x7f7e100d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e15033640 1 --2- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e1006ecb0 0x7f7e100cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e0f7fe640 1 -- 172.21.15.31:0/3198322965 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426724501 0 0) 0x7f7e100d0040 con 0x7f7e100cee00 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e0f7fe640 1 -- 172.21.15.31:0/3198322965 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e00003520 con 0x7f7e100cee00 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e14832640 1 -- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e1006ecb0 msgr2=0x7f7e100cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e14832640 1 --2- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e1006ecb0 0x7f7e100cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e14832640 1 -- 172.21.15.31:0/3198322965 >> v1:172.21.15.31:6789/0 conn(0x7f7e100cee00 legacy=0x7f7e1015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e14832640 1 -- 172.21.15.31:0/3198322965 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100cbae0 con 0x7f7e1006e8d0 2024-01-19T04:55:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.095+0000 7f7e15033640 1 --2- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e1006ecb0 0x7f7e100cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e14832640 1 --2- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 0x7f7e100d08a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097a0 tx=0x7f7e0402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856e4529ff46d08d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e0f7fe640 1 -- 172.21.15.31:0/3198322965 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0402e070 con 0x7f7e1006e8d0 2024-01-19T04:55:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e0f7fe640 1 -- 172.21.15.31:0/3198322965 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e040315a0 con 0x7f7e1006e8d0 2024-01-19T04:55:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e0f7fe640 1 -- 172.21.15.31:0/3198322965 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04031910 con 0x7f7e1006e8d0 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e16a96640 1 -- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 msgr2=0x7f7e100d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e16a96640 1 --2- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 0x7f7e100d08a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097a0 tx=0x7f7e0402b840 comp rx=0 tx=0).stop 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e16a96640 1 -- 172.21.15.31:0/3198322965 shutdown_connections 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e16a96640 1 --2- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e1006ecb0 0x7f7e100cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.096+0000 7f7e16a96640 1 --2- 172.21.15.31:0/3198322965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 0x7f7e100d08a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.097+0000 7f7e16a96640 1 -- 172.21.15.31:0/3198322965 >> 172.21.15.31:0/3198322965 conn(0x7f7e100b2c60 msgr2=0x7f7e100b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.097+0000 7f7e16a96640 1 -- 172.21.15.31:0/3198322965 shutdown_connections 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.097+0000 7f7e16a96640 1 -- 172.21.15.31:0/3198322965 wait complete. 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.098+0000 7f7e16a96640 1 Processor -- start 2024-01-19T04:55:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.098+0000 7f7e16a96640 1 -- start start 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 0x7f7e101556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e1006ecb0 0x7f7e1006c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e100cee00 0x7f7e10156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1006c370 con 0x7f7e1006ecb0 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e10155c30 con 0x7f7e1006e8d0 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1006b760 con 0x7f7e100cee00 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e1006ecb0 0x7f7e1006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e1006ecb0 0x7f7e1006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43120/0 (socket says 172.21.15.31:43120) 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 -- 172.21.15.31:0/4024651661 learned_addr learned my addr 172.21.15.31:0/4024651661 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e14832640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 0x7f7e101556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e15033640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e100cee00 0x7f7e10156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 -- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e100cee00 msgr2=0x7f7e10156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e100cee00 0x7f7e10156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 -- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 msgr2=0x7f7e101556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 0x7f7e101556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 -- 172.21.15.31:0/4024651661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08009a70 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0ffff640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e1006ecb0 0x7f7e1006c800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00c400 tx=0x7f7dfc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0d7fa640 1 -- 172.21.15.31:0/4024651661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc005ce0 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0d7fa640 1 -- 172.21.15.31:0/4024651661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc005e60 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e04009400 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e0d7fa640 1 -- 172.21.15.31:0/4024651661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0072f0 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.099+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1006bd20 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.100+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e10074a80 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.101+0000 7f7e0d7fa640 1 -- 172.21.15.31:0/4024651661 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7dfc007450 con 0x7f7e1006ecb0 2024-01-19T04:55:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.101+0000 7f7e0d7fa640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7df406aed0 0x7f7df406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.101+0000 7f7e0d7fa640 1 -- 172.21.15.31:0/4024651661 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dfc088bc0 con 0x7f7e1006ecb0 2024-01-19T04:55:31.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.101+0000 7f7e14832640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7df406aed0 0x7f7df406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.102+0000 7f7e14832640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7df406aed0 0x7f7df406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e04036000 tx=0x7f7e0403d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.105+0000 7f7e0d7fa640 1 -- 172.21.15.31:0/4024651661 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7dfc053ab0 con 0x7f7e1006ecb0 2024-01-19T04:55:31.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.279+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e1006f160 con 0x7f7df406aed0 2024-01-19T04:55:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.286+0000 7f7e0d7fa640 1 -- 172.21.15.31:0/4024651661 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264621 (secure 0 0 0) 0x7f7e040319f0 con 0x7f7df406aed0 2024-01-19T04:55:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7df406aed0 msgr2=0x7f7df406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7df406aed0 0x7f7df406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e04036000 tx=0x7f7e0403d040 comp rx=0 tx=0).stop 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e1006ecb0 msgr2=0x7f7e1006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e1006ecb0 0x7f7e1006c800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00c400 tx=0x7f7dfc00c800 comp rx=0 tx=0).stop 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 shutdown_connections 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7df406aed0 0x7f7df406d390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e100cee00 0x7f7e10156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e1006ecb0 0x7f7e1006c800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 --2- 172.21.15.31:0/4024651661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e1006e8d0 0x7f7e101556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 >> 172.21.15.31:0/4024651661 conn(0x7f7e100b2c60 msgr2=0x7f7e100b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 shutdown_connections 2024-01-19T04:55:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.289+0000 7f7e16a96640 1 -- 172.21.15.31:0/4024651661 wait complete. 2024-01-19T04:55:31.299 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-19T04:55:29.563865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219036,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002708"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671430+0000","last_change":"2024-01-19T04:55:19.508084+0000","last_active":"2024-01-19T04:55:20.671430+0000","last_peered":"2024-01-19T04:55:20.671430+0000","last_clean":"2024-01-19T04:55:20.671430+0000","last_became_active":"2024-01-19T04:55:19.507540+0000","last_became_peered":"2024-01-19T04:55:19.507540+0000","last_unstale":"2024-01-19T04:55:20.671430+0000","last_undegraded":"2024-01-19T04:55:20.671430+0000","last_fullsized":"2024-01-19T04:55:20.671430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:05.071465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491274+0000","last_change":"2024-01-19T04:55:19.503341+0000","last_active":"2024-01-19T04:55:20.491274+0000","last_peered":"2024-01-19T04:55:20.491274+0000","last_clean":"2024-01-19T04:55:20.491274+0000","last_became_active":"2024-01-19T04:55:19.503039+0000","last_became_peered":"2024-01-19T04:55:19.503039+0000","last_unstale":"2024-01-19T04:55:20.491274+0000","last_undegraded":"2024-01-19T04:55:20.491274+0000","last_fullsized":"2024-01-19T04:55:20.491274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825350+0000","last_change":"2024-01-19T04:55:19.498911+0000","last_active":"2024-01-19T04:55:20.825350+0000","last_peered":"2024-01-19T04:55:20.825350+0000","last_clean":"2024-01-19T04:55:20.825350+0000","last_became_active":"2024-01-19T04:55:19.498070+0000","last_became_peered":"2024-01-19T04:55:19.498070+0000","last_unstale":"2024-01-19T04:55:20.825350+0000","last_undegraded":"2024-01-19T04:55:20.825350+0000","last_fullsized":"2024-01-19T04:55:20.825350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492501+0000","last_change":"2024-01-19T04:55:19.501815+0000","last_active":"2024-01-19T04:55:20.492501+0000","last_peered":"2024-01-19T04:55:20.492501+0000","last_clean":"2024-01-19T04:55:20.492501+0000","last_became_active":"2024-01-19T04:55:19.501554+0000","last_became_peered":"2024-01-19T04:55:19.501554+0000","last_unstale":"2024-01-19T04:55:20.492501+0000","last_undegraded":"2024-01-19T04:55:20.492501+0000","last_fullsized":"2024-01-19T04:55:20.492501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508864+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.507962+0000","last_became_peered":"2024-01-19T04:55:19.507962+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492193+0000","last_change":"2024-01-19T04:55:19.504292+0000","last_active":"2024-01-19T04:55:20.492193+0000","last_peered":"2024-01-19T04:55:20.492193+0000","last_clean":"2024-01-19T04:55:20.492193+0000","last_became_active":"2024-01-19T04:55:19.504021+0000","last_became_peered":"2024-01-19T04:55:19.504021+0000","last_unstale":"2024-01-19T04:55:20.492193+0000","last_undegraded":"2024-01-19T04:55:20.492193+0000","last_fullsized":"2024-01-19T04:55:20.492193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670963+0000","last_change":"2024-01-19T04:55:19.507244+0000","last_active":"2024-01-19T04:55:20.670963+0000","last_peered":"2024-01-19T04:55:20.670963+0000","last_clean":"2024-01-19T04:55:20.670963+0000","last_became_active":"2024-01-19T04:55:19.506995+0000","last_became_peered":"2024-01-19T04:55:19.506995+0000","last_unstale":"2024-01-19T04:55:20.670963+0000","last_undegraded":"2024-01-19T04:55:20.670963+0000","last_fullsized":"2024-01-19T04:55:20.670963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973302+0000","last_change":"2024-01-19T04:55:19.505311+0000","last_active":"2024-01-19T04:55:20.973302+0000","last_peered":"2024-01-19T04:55:20.973302+0000","last_clean":"2024-01-19T04:55:20.973302+0000","last_became_active":"2024-01-19T04:55:19.504534+0000","last_became_peered":"2024-01-19T04:55:19.504534+0000","last_unstale":"2024-01-19T04:55:20.973302+0000","last_undegraded":"2024-01-19T04:55:20.973302+0000","last_fullsized":"2024-01-19T04:55:20.973302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671451+0000","last_change":"2024-01-19T04:55:19.508793+0000","last_active":"2024-01-19T04:55:20.671451+0000","last_peered":"2024-01-19T04:55:20.671451+0000","last_clean":"2024-01-19T04:55:20.671451+0000","last_became_active":"2024-01-19T04:55:19.508267+0000","last_became_peered":"2024-01-19T04:55:19.508267+0000","last_unstale":"2024-01-19T04:55:20.671451+0000","last_undegraded":"2024-01-19T04:55:20.671451+0000","last_fullsized":"2024-01-19T04:55:20.671451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:49:04.554504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973577+0000","last_change":"2024-01-19T04:55:19.506536+0000","last_active":"2024-01-19T04:55:20.973577+0000","last_peered":"2024-01-19T04:55:20.973577+0000","last_clean":"2024-01-19T04:55:20.973577+0000","last_became_active":"2024-01-19T04:55:19.506239+0000","last_became_peered":"2024-01-19T04:55:19.506239+0000","last_unstale":"2024-01-19T04:55:20.973577+0000","last_undegraded":"2024-01-19T04:55:20.973577+0000","last_fullsized":"2024-01-19T04:55:20.973577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973471+0000","last_change":"2024-01-19T04:55:19.502406+0000","last_active":"2024-01-19T04:55:20.973471+0000","last_peered":"2024-01-19T04:55:20.973471+0000","last_clean":"2024-01-19T04:55:20.973471+0000","last_became_active":"2024-01-19T04:55:19.502144+0000","last_became_peered":"2024-01-19T04:55:19.502144+0000","last_unstale":"2024-01-19T04:55:20.973471+0000","last_undegraded":"2024-01-19T04:55:20.973471+0000","last_fullsized":"2024-01-19T04:55:20.973471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493066+0000","last_change":"2024-01-19T04:55:19.503258+0000","last_active":"2024-01-19T04:55:20.493066+0000","last_peered":"2024-01-19T04:55:20.493066+0000","last_clean":"2024-01-19T04:55:20.493066+0000","last_became_active":"2024-01-19T04:55:19.503006+0000","last_became_peered":"2024-01-19T04:55:19.503006+0000","last_unstale":"2024-01-19T04:55:20.493066+0000","last_undegraded":"2024-01-19T04:55:20.493066+0000","last_fullsized":"2024-01-19T04:55:20.493066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973473+0000","last_change":"2024-01-19T04:55:19.507886+0000","last_active":"2024-01-19T04:55:20.973473+0000","last_peered":"2024-01-19T04:55:20.973473+0000","last_clean":"2024-01-19T04:55:20.973473+0000","last_became_active":"2024-01-19T04:55:19.507443+0000","last_became_peered":"2024-01-19T04:55:19.507443+0000","last_unstale":"2024-01-19T04:55:20.973473+0000","last_undegraded":"2024-01-19T04:55:20.973473+0000","last_fullsized":"2024-01-19T04:55:20.973473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825087+0000","last_change":"2024-01-19T04:55:19.502972+0000","last_active":"2024-01-19T04:55:20.825087+0000","last_peered":"2024-01-19T04:55:20.825087+0000","last_clean":"2024-01-19T04:55:20.825087+0000","last_became_active":"2024-01-19T04:55:19.502612+0000","last_became_peered":"2024-01-19T04:55:19.502612+0000","last_unstale":"2024-01-19T04:55:20.825087+0000","last_undegraded":"2024-01-19T04:55:20.825087+0000","last_fullsized":"2024-01-19T04:55:20.825087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:13.926376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491125+0000","last_change":"2024-01-19T04:55:19.503794+0000","last_active":"2024-01-19T04:55:20.491125+0000","last_peered":"2024-01-19T04:55:20.491125+0000","last_clean":"2024-01-19T04:55:20.491125+0000","last_became_active":"2024-01-19T04:55:19.503516+0000","last_became_peered":"2024-01-19T04:55:19.503516+0000","last_unstale":"2024-01-19T04:55:20.491125+0000","last_undegraded":"2024-01-19T04:55:20.491125+0000","last_fullsized":"2024-01-19T04:55:20.491125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491667+0000","last_change":"2024-01-19T04:55:19.500410+0000","last_active":"2024-01-19T04:55:20.491667+0000","last_peered":"2024-01-19T04:55:20.491667+0000","last_clean":"2024-01-19T04:55:20.491667+0000","last_became_active":"2024-01-19T04:55:19.500147+0000","last_became_peered":"2024-01-19T04:55:19.500147+0000","last_unstale":"2024-01-19T04:55:20.491667+0000","last_undegraded":"2024-01-19T04:55:20.491667+0000","last_fullsized":"2024-01-19T04:55:20.491667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493061+0000","last_change":"2024-01-19T04:55:19.507266+0000","last_active":"2024-01-19T04:55:20.493061+0000","last_peered":"2024-01-19T04:55:20.493061+0000","last_clean":"2024-01-19T04:55:20.493061+0000","last_became_active":"2024-01-19T04:55:19.506970+0000","last_became_peered":"2024-01-19T04:55:19.506970+0000","last_unstale":"2024-01-19T04:55:20.493061+0000","last_undegraded":"2024-01-19T04:55:20.493061+0000","last_fullsized":"2024-01-19T04:55:20.493061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973133+0000","last_change":"2024-01-19T04:55:19.501787+0000","last_active":"2024-01-19T04:55:20.973133+0000","last_peered":"2024-01-19T04:55:20.973133+0000","last_clean":"2024-01-19T04:55:20.973133+0000","last_became_active":"2024-01-19T04:55:19.501389+0000","last_became_peered":"2024-01-19T04:55:19.501389+0000","last_unstale":"2024-01-19T04:55:20.973133+0000","last_undegraded":"2024-01-19T04:55:20.973133+0000","last_fullsized":"2024-01-19T04:55:20.973133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671369+0000","last_change":"2024-01-19T04:55:19.507131+0000","last_active":"2024-01-19T04:55:20.671369+0000","last_peered":"2024-01-19T04:55:20.671369+0000","last_clean":"2024-01-19T04:55:20.671369+0000","last_became_active":"2024-01-19T04:55:19.506871+0000","last_became_peered":"2024-01-19T04:55:19.506871+0000","last_unstale":"2024-01-19T04:55:20.671369+0000","last_undegraded":"2024-01-19T04:55:20.671369+0000","last_fullsized":"2024-01-19T04:55:20.671369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:29:51.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973517+0000","last_change":"2024-01-19T04:55:19.504339+0000","last_active":"2024-01-19T04:55:20.973517+0000","last_peered":"2024-01-19T04:55:20.973517+0000","last_clean":"2024-01-19T04:55:20.973517+0000","last_became_active":"2024-01-19T04:55:19.504087+0000","last_became_peered":"2024-01-19T04:55:19.504087+0000","last_unstale":"2024-01-19T04:55:20.973517+0000","last_undegraded":"2024-01-19T04:55:20.973517+0000","last_fullsized":"2024-01-19T04:55:20.973517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825223+0000","last_change":"2024-01-19T04:55:19.510243+0000","last_active":"2024-01-19T04:55:20.825223+0000","last_peered":"2024-01-19T04:55:20.825223+0000","last_clean":"2024-01-19T04:55:20.825223+0000","last_became_active":"2024-01-19T04:55:19.509971+0000","last_became_peered":"2024-01-19T04:55:19.509971+0000","last_unstale":"2024-01-19T04:55:20.825223+0000","last_undegraded":"2024-01-19T04:55:20.825223+0000","last_fullsized":"2024-01-19T04:55:20.825223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492345+0000","last_change":"2024-01-19T04:55:19.509452+0000","last_active":"2024-01-19T04:55:20.492345+0000","last_peered":"2024-01-19T04:55:20.492345+0000","last_clean":"2024-01-19T04:55:20.492345+0000","last_became_active":"2024-01-19T04:55:19.507159+0000","last_became_peered":"2024-01-19T04:55:19.507159+0000","last_unstale":"2024-01-19T04:55:20.492345+0000","last_undegraded":"2024-01-19T04:55:20.492345+0000","last_fullsized":"2024-01-19T04:55:20.492345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973182+0000","last_change":"2024-01-19T04:55:19.508179+0000","last_active":"2024-01-19T04:55:20.973182+0000","last_peered":"2024-01-19T04:55:20.973182+0000","last_clean":"2024-01-19T04:55:20.973182+0000","last_became_active":"2024-01-19T04:55:19.507867+0000","last_became_peered":"2024-01-19T04:55:19.507867+0000","last_unstale":"2024-01-19T04:55:20.973182+0000","last_undegraded":"2024-01-19T04:55:20.973182+0000","last_fullsized":"2024-01-19T04:55:20.973182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491882+0000","last_change":"2024-01-19T04:55:19.500977+0000","last_active":"2024-01-19T04:55:20.491882+0000","last_peered":"2024-01-19T04:55:20.491882+0000","last_clean":"2024-01-19T04:55:20.491882+0000","last_became_active":"2024-01-19T04:55:19.500722+0000","last_became_peered":"2024-01-19T04:55:19.500722+0000","last_unstale":"2024-01-19T04:55:20.491882+0000","last_undegraded":"2024-01-19T04:55:20.491882+0000","last_fullsized":"2024-01-19T04:55:20.491882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973622+0000","last_change":"2024-01-19T04:55:19.505026+0000","last_active":"2024-01-19T04:55:20.973622+0000","last_peered":"2024-01-19T04:55:20.973622+0000","last_clean":"2024-01-19T04:55:20.973622+0000","last_became_active":"2024-01-19T04:55:19.504732+0000","last_became_peered":"2024-01-19T04:55:19.504732+0000","last_unstale":"2024-01-19T04:55:20.973622+0000","last_undegraded":"2024-01-19T04:55:20.973622+0000","last_fullsized":"2024-01-19T04:55:20.973622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973617+0000","last_change":"2024-01-19T04:55:19.508736+0000","last_active":"2024-01-19T04:55:20.973617+0000","last_peered":"2024-01-19T04:55:20.973617+0000","last_clean":"2024-01-19T04:55:20.973617+0000","last_became_active":"2024-01-19T04:55:19.508432+0000","last_became_peered":"2024-01-19T04:55:19.508432+0000","last_unstale":"2024-01-19T04:55:20.973617+0000","last_undegraded":"2024-01-19T04:55:20.973617+0000","last_fullsized":"2024-01-19T04:55:20.973617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491221+0000","last_change":"2024-01-19T04:55:19.508491+0000","last_active":"2024-01-19T04:55:20.491221+0000","last_peered":"2024-01-19T04:55:20.491221+0000","last_clean":"2024-01-19T04:55:20.491221+0000","last_became_active":"2024-01-19T04:55:19.508174+0000","last_became_peered":"2024-01-19T04:55:19.508174+0000","last_unstale":"2024-01-19T04:55:20.491221+0000","last_undegraded":"2024-01-19T04:55:20.491221+0000","last_fullsized":"2024-01-19T04:55:20.491221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973603+0000","last_change":"2024-01-19T04:55:19.506159+0000","last_active":"2024-01-19T04:55:20.973603+0000","last_peered":"2024-01-19T04:55:20.973603+0000","last_clean":"2024-01-19T04:55:20.973603+0000","last_became_active":"2024-01-19T04:55:19.505854+0000","last_became_peered":"2024-01-19T04:55:19.505854+0000","last_unstale":"2024-01-19T04:55:20.973603+0000","last_undegraded":"2024-01-19T04:55:20.973603+0000","last_fullsized":"2024-01-19T04:55:20.973603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825356+0000","last_change":"2024-01-19T04:55:19.509241+0000","last_active":"2024-01-19T04:55:20.825356+0000","last_peered":"2024-01-19T04:55:20.825356+0000","last_clean":"2024-01-19T04:55:20.825356+0000","last_became_active":"2024-01-19T04:55:19.508899+0000","last_became_peered":"2024-01-19T04:55:19.508899+0000","last_unstale":"2024-01-19T04:55:20.825356+0000","last_undegraded":"2024-01-19T04:55:20.825356+0000","last_fullsized":"2024-01-19T04:55:20.825356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492316+0000","last_change":"2024-01-19T04:55:19.505518+0000","last_active":"2024-01-19T04:55:20.492316+0000","last_peered":"2024-01-19T04:55:20.492316+0000","last_clean":"2024-01-19T04:55:20.492316+0000","last_became_active":"2024-01-19T04:55:19.505203+0000","last_became_peered":"2024-01-19T04:55:19.505203+0000","last_unstale":"2024-01-19T04:55:20.492316+0000","last_undegraded":"2024-01-19T04:55:20.492316+0000","last_fullsized":"2024-01-19T04:55:20.492316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973044+0000","last_change":"2024-01-19T04:55:19.500467+0000","last_active":"2024-01-19T04:55:20.973044+0000","last_peered":"2024-01-19T04:55:20.973044+0000","last_clean":"2024-01-19T04:55:20.973044+0000","last_became_active":"2024-01-19T04:55:19.500146+0000","last_became_peered":"2024-01-19T04:55:19.500146+0000","last_unstale":"2024-01-19T04:55:20.973044+0000","last_undegraded":"2024-01-19T04:55:20.973044+0000","last_fullsized":"2024-01-19T04:55:20.973044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973209+0000","last_change":"2024-01-19T04:55:19.504033+0000","last_active":"2024-01-19T04:55:20.973209+0000","last_peered":"2024-01-19T04:55:20.973209+0000","last_clean":"2024-01-19T04:55:20.973209+0000","last_became_active":"2024-01-19T04:55:19.503555+0000","last_became_peered":"2024-01-19T04:55:19.503555+0000","last_unstale":"2024-01-19T04:55:20.973209+0000","last_undegraded":"2024-01-19T04:55:20.973209+0000","last_fullsized":"2024-01-19T04:55:20.973209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492127+0000","last_change":"2024-01-19T04:55:19.505336+0000","last_active":"2024-01-19T04:55:20.492127+0000","last_peered":"2024-01-19T04:55:20.492127+0000","last_clean":"2024-01-19T04:55:20.492127+0000","last_became_active":"2024-01-19T04:55:19.505035+0000","last_became_peered":"2024-01-19T04:55:19.505035+0000","last_unstale":"2024-01-19T04:55:20.492127+0000","last_undegraded":"2024-01-19T04:55:20.492127+0000","last_fullsized":"2024-01-19T04:55:20.492127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:38.193951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973572+0000","last_change":"2024-01-19T04:55:18.533807+0000","last_active":"2024-01-19T04:55:20.973572+0000","last_peered":"2024-01-19T04:55:20.973572+0000","last_clean":"2024-01-19T04:55:20.973572+0000","last_became_active":"2024-01-19T04:55:18.533426+0000","last_became_peered":"2024-01-19T04:55:18.533426+0000","last_unstale":"2024-01-19T04:55:20.973572+0000","last_undegraded":"2024-01-19T04:55:20.973572+0000","last_fullsized":"2024-01-19T04:55:20.973572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825363+0000","last_change":"2024-01-19T04:55:19.502475+0000","last_active":"2024-01-19T04:55:20.825363+0000","last_peered":"2024-01-19T04:55:20.825363+0000","last_clean":"2024-01-19T04:55:20.825363+0000","last_became_active":"2024-01-19T04:55:19.502036+0000","last_became_peered":"2024-01-19T04:55:19.502036+0000","last_unstale":"2024-01-19T04:55:20.825363+0000","last_undegraded":"2024-01-19T04:55:20.825363+0000","last_fullsized":"2024-01-19T04:55:20.825363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973270+0000","last_change":"2024-01-19T04:55:18.525394+0000","last_active":"2024-01-19T04:55:20.973270+0000","last_peered":"2024-01-19T04:55:20.973270+0000","last_clean":"2024-01-19T04:55:20.973270+0000","last_became_active":"2024-01-19T04:55:18.525118+0000","last_became_peered":"2024-01-19T04:55:18.525118+0000","last_unstale":"2024-01-19T04:55:20.973270+0000","last_undegraded":"2024-01-19T04:55:20.973270+0000","last_fullsized":"2024-01-19T04:55:20.973270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973314+0000","last_change":"2024-01-19T04:55:18.526820+0000","last_active":"2024-01-19T04:55:20.973314+0000","last_peered":"2024-01-19T04:55:20.973314+0000","last_clean":"2024-01-19T04:55:20.973314+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.973314+0000","last_undegraded":"2024-01-19T04:55:20.973314+0000","last_fullsized":"2024-01-19T04:55:20.973314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973308+0000","last_change":"2024-01-19T04:55:19.505074+0000","last_active":"2024-01-19T04:55:20.973308+0000","last_peered":"2024-01-19T04:55:20.973308+0000","last_clean":"2024-01-19T04:55:20.973308+0000","last_became_active":"2024-01-19T04:55:19.504093+0000","last_became_peered":"2024-01-19T04:55:19.504093+0000","last_unstale":"2024-01-19T04:55:20.973308+0000","last_undegraded":"2024-01-19T04:55:20.973308+0000","last_fullsized":"2024-01-19T04:55:20.973308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491874+0000","last_change":"2024-01-19T04:55:18.520981+0000","last_active":"2024-01-19T04:55:20.491874+0000","last_peered":"2024-01-19T04:55:20.491874+0000","last_clean":"2024-01-19T04:55:20.491874+0000","last_became_active":"2024-01-19T04:55:18.520110+0000","last_became_peered":"2024-01-19T04:55:18.520110+0000","last_unstale":"2024-01-19T04:55:20.491874+0000","last_undegraded":"2024-01-19T04:55:20.491874+0000","last_fullsized":"2024-01-19T04:55:20.491874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491068+0000","last_change":"2024-01-19T04:55:19.510473+0000","last_active":"2024-01-19T04:55:20.491068+0000","last_peered":"2024-01-19T04:55:20.491068+0000","last_clean":"2024-01-19T04:55:20.491068+0000","last_became_active":"2024-01-19T04:55:19.510207+0000","last_became_peered":"2024-01-19T04:55:19.510207+0000","last_unstale":"2024-01-19T04:55:20.491068+0000","last_undegraded":"2024-01-19T04:55:20.491068+0000","last_fullsized":"2024-01-19T04:55:20.491068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973574+0000","last_change":"2024-01-19T04:55:18.528905+0000","last_active":"2024-01-19T04:55:20.973574+0000","last_peered":"2024-01-19T04:55:20.973574+0000","last_clean":"2024-01-19T04:55:20.973574+0000","last_became_active":"2024-01-19T04:55:18.528580+0000","last_became_peered":"2024-01-19T04:55:18.528580+0000","last_unstale":"2024-01-19T04:55:20.973574+0000","last_undegraded":"2024-01-19T04:55:20.973574+0000","last_fullsized":"2024-01-19T04:55:20.973574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492708+0000","last_change":"2024-01-19T04:55:19.509724+0000","last_active":"2024-01-19T04:55:20.492708+0000","last_peered":"2024-01-19T04:55:20.492708+0000","last_clean":"2024-01-19T04:55:20.492708+0000","last_became_active":"2024-01-19T04:55:19.509431+0000","last_became_peered":"2024-01-19T04:55:19.509431+0000","last_unstale":"2024-01-19T04:55:20.492708+0000","last_undegraded":"2024-01-19T04:55:20.492708+0000","last_fullsized":"2024-01-19T04:55:20.492708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492870+0000","last_change":"2024-01-19T04:55:18.525788+0000","last_active":"2024-01-19T04:55:20.492870+0000","last_peered":"2024-01-19T04:55:20.492870+0000","last_clean":"2024-01-19T04:55:20.492870+0000","last_became_active":"2024-01-19T04:55:18.525521+0000","last_became_peered":"2024-01-19T04:55:18.525521+0000","last_unstale":"2024-01-19T04:55:20.492870+0000","last_undegraded":"2024-01-19T04:55:20.492870+0000","last_fullsized":"2024-01-19T04:55:20.492870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825497+0000","last_change":"2024-01-19T04:55:19.501054+0000","last_active":"2024-01-19T04:55:20.825497+0000","last_peered":"2024-01-19T04:55:20.825497+0000","last_clean":"2024-01-19T04:55:20.825497+0000","last_became_active":"2024-01-19T04:55:19.500559+0000","last_became_peered":"2024-01-19T04:55:19.500559+0000","last_unstale":"2024-01-19T04:55:20.825497+0000","last_undegraded":"2024-01-19T04:55:20.825497+0000","last_fullsized":"2024-01-19T04:55:20.825497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491237+0000","last_change":"2024-01-19T04:55:18.528505+0000","last_active":"2024-01-19T04:55:20.491237+0000","last_peered":"2024-01-19T04:55:20.491237+0000","last_clean":"2024-01-19T04:55:20.491237+0000","last_became_active":"2024-01-19T04:55:18.528221+0000","last_became_peered":"2024-01-19T04:55:18.528221+0000","last_unstale":"2024-01-19T04:55:20.491237+0000","last_undegraded":"2024-01-19T04:55:20.491237+0000","last_fullsized":"2024-01-19T04:55:20.491237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:22.050835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493257+0000","last_change":"2024-01-19T04:55:19.507145+0000","last_active":"2024-01-19T04:55:20.493257+0000","last_peered":"2024-01-19T04:55:20.493257+0000","last_clean":"2024-01-19T04:55:20.493257+0000","last_became_active":"2024-01-19T04:55:19.506516+0000","last_became_peered":"2024-01-19T04:55:19.506516+0000","last_unstale":"2024-01-19T04:55:20.493257+0000","last_undegraded":"2024-01-19T04:55:20.493257+0000","last_fullsized":"2024-01-19T04:55:20.493257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:42.763778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973447+0000","last_change":"2024-01-19T04:55:18.527944+0000","last_active":"2024-01-19T04:55:20.973447+0000","last_peered":"2024-01-19T04:55:20.973447+0000","last_clean":"2024-01-19T04:55:20.973447+0000","last_became_active":"2024-01-19T04:55:18.527627+0000","last_became_peered":"2024-01-19T04:55:18.527627+0000","last_unstale":"2024-01-19T04:55:20.973447+0000","last_undegraded":"2024-01-19T04:55:20.973447+0000","last_fullsized":"2024-01-19T04:55:20.973447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508912+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.508650+0000","last_became_peered":"2024-01-19T04:55:19.508650+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825436+0000","last_change":"2024-01-19T04:55:18.529432+0000","last_active":"2024-01-19T04:55:20.825436+0000","last_peered":"2024-01-19T04:55:20.825436+0000","last_clean":"2024-01-19T04:55:20.825436+0000","last_became_active":"2024-01-19T04:55:18.529166+0000","last_became_peered":"2024-01-19T04:55:18.529166+0000","last_unstale":"2024-01-19T04:55:20.825436+0000","last_undegraded":"2024-01-19T04:55:20.825436+0000","last_fullsized":"2024-01-19T04:55:20.825436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825157+0000","last_change":"2024-01-19T04:55:19.507049+0000","last_active":"2024-01-19T04:55:20.825157+0000","last_peered":"2024-01-19T04:55:20.825157+0000","last_clean":"2024-01-19T04:55:20.825157+0000","last_became_active":"2024-01-19T04:55:19.506681+0000","last_became_peered":"2024-01-19T04:55:19.506681+0000","last_unstale":"2024-01-19T04:55:20.825157+0000","last_undegraded":"2024-01-19T04:55:20.825157+0000","last_fullsized":"2024-01-19T04:55:20.825157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492787+0000","last_change":"2024-01-19T04:55:18.522952+0000","last_active":"2024-01-19T04:55:20.492787+0000","last_peered":"2024-01-19T04:55:20.492787+0000","last_clean":"2024-01-19T04:55:20.492787+0000","last_became_active":"2024-01-19T04:55:18.522594+0000","last_became_peered":"2024-01-19T04:55:18.522594+0000","last_unstale":"2024-01-19T04:55:20.492787+0000","last_undegraded":"2024-01-19T04:55:20.492787+0000","last_fullsized":"2024-01-19T04:55:20.492787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671150+0000","last_change":"2024-01-19T04:55:19.509175+0000","last_active":"2024-01-19T04:55:20.671150+0000","last_peered":"2024-01-19T04:55:20.671150+0000","last_clean":"2024-01-19T04:55:20.671150+0000","last_became_active":"2024-01-19T04:55:19.508249+0000","last_became_peered":"2024-01-19T04:55:19.508249+0000","last_unstale":"2024-01-19T04:55:20.671150+0000","last_undegraded":"2024-01-19T04:55:20.671150+0000","last_fullsized":"2024-01-19T04:55:20.671150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973205+0000","last_change":"2024-01-19T04:55:18.525890+0000","last_active":"2024-01-19T04:55:20.973205+0000","last_peered":"2024-01-19T04:55:20.973205+0000","last_clean":"2024-01-19T04:55:20.973205+0000","last_became_active":"2024-01-19T04:55:18.525630+0000","last_became_peered":"2024-01-19T04:55:18.525630+0000","last_unstale":"2024-01-19T04:55:20.973205+0000","last_undegraded":"2024-01-19T04:55:20.973205+0000","last_fullsized":"2024-01-19T04:55:20.973205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973278+0000","last_change":"2024-01-19T04:55:19.503820+0000","last_active":"2024-01-19T04:55:20.973278+0000","last_peered":"2024-01-19T04:55:20.973278+0000","last_clean":"2024-01-19T04:55:20.973278+0000","last_became_active":"2024-01-19T04:55:19.503248+0000","last_became_peered":"2024-01-19T04:55:19.503248+0000","last_unstale":"2024-01-19T04:55:20.973278+0000","last_undegraded":"2024-01-19T04:55:20.973278+0000","last_fullsized":"2024-01-19T04:55:20.973278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973005+0000","last_change":"2024-01-19T04:55:18.521372+0000","last_active":"2024-01-19T04:55:20.973005+0000","last_peered":"2024-01-19T04:55:20.973005+0000","last_clean":"2024-01-19T04:55:20.973005+0000","last_became_active":"2024-01-19T04:55:18.521021+0000","last_became_peered":"2024-01-19T04:55:18.521021+0000","last_unstale":"2024-01-19T04:55:20.973005+0000","last_undegraded":"2024-01-19T04:55:20.973005+0000","last_fullsized":"2024-01-19T04:55:20.973005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671050+0000","last_change":"2024-01-19T04:55:19.507703+0000","last_active":"2024-01-19T04:55:20.671050+0000","last_peered":"2024-01-19T04:55:20.671050+0000","last_clean":"2024-01-19T04:55:20.671050+0000","last_became_active":"2024-01-19T04:55:19.507243+0000","last_became_peered":"2024-01-19T04:55:19.507243+0000","last_unstale":"2024-01-19T04:55:20.671050+0000","last_undegraded":"2024-01-19T04:55:20.671050+0000","last_fullsized":"2024-01-19T04:55:20.671050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492777+0000","last_change":"2024-01-19T04:55:18.531093+0000","last_active":"2024-01-19T04:55:20.492777+0000","last_peered":"2024-01-19T04:55:20.492777+0000","last_clean":"2024-01-19T04:55:20.492777+0000","last_became_active":"2024-01-19T04:55:18.530783+0000","last_became_peered":"2024-01-19T04:55:18.530783+0000","last_unstale":"2024-01-19T04:55:20.492777+0000","last_undegraded":"2024-01-19T04:55:20.492777+0000","last_fullsized":"2024-01-19T04:55:20.492777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:18:16.876671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671483+0000","last_change":"2024-01-19T04:55:19.506668+0000","last_active":"2024-01-19T04:55:20.671483+0000","last_peered":"2024-01-19T04:55:20.671483+0000","last_clean":"2024-01-19T04:55:20.671483+0000","last_became_active":"2024-01-19T04:55:19.506350+0000","last_became_peered":"2024-01-19T04:55:19.506350+0000","last_unstale":"2024-01-19T04:55:20.671483+0000","last_undegraded":"2024-01-19T04:55:20.671483+0000","last_fullsized":"2024-01-19T04:55:20.671483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:54:10.730504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973247+0000","last_change":"2024-01-19T04:55:18.523120+0000","last_active":"2024-01-19T04:55:20.973247+0000","last_peered":"2024-01-19T04:55:20.973247+0000","last_clean":"2024-01-19T04:55:20.973247+0000","last_became_active":"2024-01-19T04:55:18.522810+0000","last_became_peered":"2024-01-19T04:55:18.522810+0000","last_unstale":"2024-01-19T04:55:20.973247+0000","last_undegraded":"2024-01-19T04:55:20.973247+0000","last_fullsized":"2024-01-19T04:55:20.973247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671581+0000","last_change":"2024-01-19T04:55:19.508309+0000","last_active":"2024-01-19T04:55:20.671581+0000","last_peered":"2024-01-19T04:55:20.671581+0000","last_clean":"2024-01-19T04:55:20.671581+0000","last_became_active":"2024-01-19T04:55:19.507838+0000","last_became_peered":"2024-01-19T04:55:19.507838+0000","last_unstale":"2024-01-19T04:55:20.671581+0000","last_undegraded":"2024-01-19T04:55:20.671581+0000","last_fullsized":"2024-01-19T04:55:20.671581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:59:10.745652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491300+0000","last_change":"2024-01-19T04:55:18.526216+0000","last_active":"2024-01-19T04:55:20.491300+0000","last_peered":"2024-01-19T04:55:20.491300+0000","last_clean":"2024-01-19T04:55:20.491300+0000","last_became_active":"2024-01-19T04:55:18.524313+0000","last_became_peered":"2024-01-19T04:55:18.524313+0000","last_unstale":"2024-01-19T04:55:20.491300+0000","last_undegraded":"2024-01-19T04:55:20.491300+0000","last_fullsized":"2024-01-19T04:55:20.491300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492661+0000","last_change":"2024-01-19T04:55:19.501625+0000","last_active":"2024-01-19T04:55:20.492661+0000","last_peered":"2024-01-19T04:55:20.492661+0000","last_clean":"2024-01-19T04:55:20.492661+0000","last_became_active":"2024-01-19T04:55:19.500107+0000","last_became_peered":"2024-01-19T04:55:19.500107+0000","last_unstale":"2024-01-19T04:55:20.492661+0000","last_undegraded":"2024-01-19T04:55:20.492661+0000","last_fullsized":"2024-01-19T04:55:20.492661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973138+0000","last_change":"2024-01-19T04:55:18.525457+0000","last_active":"2024-01-19T04:55:20.973138+0000","last_peered":"2024-01-19T04:55:20.973138+0000","last_clean":"2024-01-19T04:55:20.973138+0000","last_became_active":"2024-01-19T04:55:18.525183+0000","last_became_peered":"2024-01-19T04:55:18.525183+0000","last_unstale":"2024-01-19T04:55:20.973138+0000","last_undegraded":"2024-01-19T04:55:20.973138+0000","last_fullsized":"2024-01-19T04:55:20.973138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671198+0000","last_change":"2024-01-19T04:55:19.508457+0000","last_active":"2024-01-19T04:55:20.671198+0000","last_peered":"2024-01-19T04:55:20.671198+0000","last_clean":"2024-01-19T04:55:20.671198+0000","last_became_active":"2024-01-19T04:55:19.506325+0000","last_became_peered":"2024-01-19T04:55:19.506325+0000","last_unstale":"2024-01-19T04:55:20.671198+0000","last_undegraded":"2024-01-19T04:55:20.671198+0000","last_fullsized":"2024-01-19T04:55:20.671198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:01:52.567337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-19T04:55:19.496795+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:55:19.496795+0000","last_peered":"2024-01-19T04:55:19.496795+0000","last_clean":"2024-01-19T04:55:19.496795+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:55:19.496795+0000","last_undegraded":"2024-01-19T04:55:19.496795+0000","last_fullsized":"2024-01-19T04:55:19.496795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492187+0000","last_change":"2024-01-19T04:55:18.520709+0000","last_active":"2024-01-19T04:55:20.492187+0000","last_peered":"2024-01-19T04:55:20.492187+0000","last_clean":"2024-01-19T04:55:20.492187+0000","last_became_active":"2024-01-19T04:55:18.520257+0000","last_became_peered":"2024-01-19T04:55:18.520257+0000","last_unstale":"2024-01-19T04:55:20.492187+0000","last_undegraded":"2024-01-19T04:55:20.492187+0000","last_fullsized":"2024-01-19T04:55:20.492187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973410+0000","last_change":"2024-01-19T04:55:19.507460+0000","last_active":"2024-01-19T04:55:20.973410+0000","last_peered":"2024-01-19T04:55:20.973410+0000","last_clean":"2024-01-19T04:55:20.973410+0000","last_became_active":"2024-01-19T04:55:19.507182+0000","last_became_peered":"2024-01-19T04:55:19.507182+0000","last_unstale":"2024-01-19T04:55:20.973410+0000","last_undegraded":"2024-01-19T04:55:20.973410+0000","last_fullsized":"2024-01-19T04:55:20.973410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670846+0000","last_change":"2024-01-19T04:55:18.531158+0000","last_active":"2024-01-19T04:55:20.670846+0000","last_peered":"2024-01-19T04:55:20.670846+0000","last_clean":"2024-01-19T04:55:20.670846+0000","last_became_active":"2024-01-19T04:55:18.530811+0000","last_became_peered":"2024-01-19T04:55:18.530811+0000","last_unstale":"2024-01-19T04:55:20.670846+0000","last_undegraded":"2024-01-19T04:55:20.670846+0000","last_fullsized":"2024-01-19T04:55:20.670846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:31.580264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825075+0000","last_change":"2024-01-19T04:55:19.503305+0000","last_active":"2024-01-19T04:55:20.825075+0000","last_peered":"2024-01-19T04:55:20.825075+0000","last_clean":"2024-01-19T04:55:20.825075+0000","last_became_active":"2024-01-19T04:55:19.502993+0000","last_became_peered":"2024-01-19T04:55:19.502993+0000","last_unstale":"2024-01-19T04:55:20.825075+0000","last_undegraded":"2024-01-19T04:55:20.825075+0000","last_fullsized":"2024-01-19T04:55:20.825075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:32.585555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973077+0000","last_change":"2024-01-19T04:55:18.523147+0000","last_active":"2024-01-19T04:55:20.973077+0000","last_peered":"2024-01-19T04:55:20.973077+0000","last_clean":"2024-01-19T04:55:20.973077+0000","last_became_active":"2024-01-19T04:55:18.522848+0000","last_became_peered":"2024-01-19T04:55:18.522848+0000","last_unstale":"2024-01-19T04:55:20.973077+0000","last_undegraded":"2024-01-19T04:55:20.973077+0000","last_fullsized":"2024-01-19T04:55:20.973077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491259+0000","last_change":"2024-01-19T04:55:19.500588+0000","last_active":"2024-01-19T04:55:20.491259+0000","last_peered":"2024-01-19T04:55:20.491259+0000","last_clean":"2024-01-19T04:55:20.491259+0000","last_became_active":"2024-01-19T04:55:19.500210+0000","last_became_peered":"2024-01-19T04:55:19.500210+0000","last_unstale":"2024-01-19T04:55:20.491259+0000","last_undegraded":"2024-01-19T04:55:20.491259+0000","last_fullsized":"2024-01-19T04:55:20.491259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492838+0000","last_change":"2024-01-19T04:55:18.521717+0000","last_active":"2024-01-19T04:55:20.492838+0000","last_peered":"2024-01-19T04:55:20.492838+0000","last_clean":"2024-01-19T04:55:20.492838+0000","last_became_active":"2024-01-19T04:55:18.521452+0000","last_became_peered":"2024-01-19T04:55:18.521452+0000","last_unstale":"2024-01-19T04:55:20.492838+0000","last_undegraded":"2024-01-19T04:55:20.492838+0000","last_fullsized":"2024-01-19T04:55:20.492838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492300+0000","last_change":"2024-01-19T04:55:19.497086+0000","last_active":"2024-01-19T04:55:20.492300+0000","last_peered":"2024-01-19T04:55:20.492300+0000","last_clean":"2024-01-19T04:55:20.492300+0000","last_became_active":"2024-01-19T04:55:19.496812+0000","last_became_peered":"2024-01-19T04:55:19.496812+0000","last_unstale":"2024-01-19T04:55:20.492300+0000","last_undegraded":"2024-01-19T04:55:20.492300+0000","last_fullsized":"2024-01-19T04:55:20.492300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:24.177458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491255+0000","last_change":"2024-01-19T04:55:18.531156+0000","last_active":"2024-01-19T04:55:20.491255+0000","last_peered":"2024-01-19T04:55:20.491255+0000","last_clean":"2024-01-19T04:55:20.491255+0000","last_became_active":"2024-01-19T04:55:18.530716+0000","last_became_peered":"2024-01-19T04:55:18.530716+0000","last_unstale":"2024-01-19T04:55:20.491255+0000","last_undegraded":"2024-01-19T04:55:20.491255+0000","last_fullsized":"2024-01-19T04:55:20.491255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492468+0000","last_change":"2024-01-19T04:55:19.497882+0000","last_active":"2024-01-19T04:55:20.492468+0000","last_peered":"2024-01-19T04:55:20.492468+0000","last_clean":"2024-01-19T04:55:20.492468+0000","last_became_active":"2024-01-19T04:55:19.497509+0000","last_became_peered":"2024-01-19T04:55:19.497509+0000","last_unstale":"2024-01-19T04:55:20.492468+0000","last_undegraded":"2024-01-19T04:55:20.492468+0000","last_fullsized":"2024-01-19T04:55:20.492468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671479+0000","last_change":"2024-01-19T04:55:18.539166+0000","last_active":"2024-01-19T04:55:20.671479+0000","last_peered":"2024-01-19T04:55:20.671479+0000","last_clean":"2024-01-19T04:55:20.671479+0000","last_became_active":"2024-01-19T04:55:18.538937+0000","last_became_peered":"2024-01-19T04:55:18.538937+0000","last_unstale":"2024-01-19T04:55:20.671479+0000","last_undegraded":"2024-01-19T04:55:20.671479+0000","last_fullsized":"2024-01-19T04:55:20.671479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:21:12.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491285+0000","last_change":"2024-01-19T04:55:19.499782+0000","last_active":"2024-01-19T04:55:20.491285+0000","last_peered":"2024-01-19T04:55:20.491285+0000","last_clean":"2024-01-19T04:55:20.491285+0000","last_became_active":"2024-01-19T04:55:19.499495+0000","last_became_peered":"2024-01-19T04:55:19.499495+0000","last_unstale":"2024-01-19T04:55:20.491285+0000","last_undegraded":"2024-01-19T04:55:20.491285+0000","last_fullsized":"2024-01-19T04:55:20.491285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:21.576294+0000","last_change":"2024-01-19T04:55:18.530049+0000","last_active":"2024-01-19T04:55:21.576294+0000","last_peered":"2024-01-19T04:55:21.576294+0000","last_clean":"2024-01-19T04:55:21.576294+0000","last_became_active":"2024-01-19T04:55:18.529767+0000","last_became_peered":"2024-01-19T04:55:18.529767+0000","last_unstale":"2024-01-19T04:55:21.576294+0000","last_undegraded":"2024-01-19T04:55:21.576294+0000","last_fullsized":"2024-01-19T04:55:21.576294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973714+0000","last_change":"2024-01-19T04:55:19.511086+0000","last_active":"2024-01-19T04:55:20.973714+0000","last_peered":"2024-01-19T04:55:20.973714+0000","last_clean":"2024-01-19T04:55:20.973714+0000","last_became_active":"2024-01-19T04:55:19.510781+0000","last_became_peered":"2024-01-19T04:55:19.510781+0000","last_unstale":"2024-01-19T04:55:20.973714+0000","last_undegraded":"2024-01-19T04:55:20.973714+0000","last_fullsized":"2024-01-19T04:55:20.973714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973635+0000","last_change":"2024-01-19T04:55:18.529646+0000","last_active":"2024-01-19T04:55:20.973635+0000","last_peered":"2024-01-19T04:55:20.973635+0000","last_clean":"2024-01-19T04:55:20.973635+0000","last_became_active":"2024-01-19T04:55:18.529403+0000","last_became_peered":"2024-01-19T04:55:18.529403+0000","last_unstale":"2024-01-19T04:55:20.973635+0000","last_undegraded":"2024-01-19T04:55:20.973635+0000","last_fullsized":"2024-01-19T04:55:20.973635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825397+0000","last_change":"2024-01-19T04:55:19.503772+0000","last_active":"2024-01-19T04:55:20.825397+0000","last_peered":"2024-01-19T04:55:20.825397+0000","last_clean":"2024-01-19T04:55:20.825397+0000","last_became_active":"2024-01-19T04:55:19.503188+0000","last_became_peered":"2024-01-19T04:55:19.503188+0000","last_unstale":"2024-01-19T04:55:20.825397+0000","last_undegraded":"2024-01-19T04:55:20.825397+0000","last_fullsized":"2024-01-19T04:55:20.825397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670816+0000","last_change":"2024-01-19T04:55:18.536711+0000","last_active":"2024-01-19T04:55:20.670816+0000","last_peered":"2024-01-19T04:55:20.670816+0000","last_clean":"2024-01-19T04:55:20.670816+0000","last_became_active":"2024-01-19T04:55:18.536483+0000","last_became_peered":"2024-01-19T04:55:18.536483+0000","last_unstale":"2024-01-19T04:55:20.670816+0000","last_undegraded":"2024-01-19T04:55:20.670816+0000","last_fullsized":"2024-01-19T04:55:20.670816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:12.921084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492909+0000","last_change":"2024-01-19T04:55:19.496953+0000","last_active":"2024-01-19T04:55:20.492909+0000","last_peered":"2024-01-19T04:55:20.492909+0000","last_clean":"2024-01-19T04:55:20.492909+0000","last_became_active":"2024-01-19T04:55:19.496452+0000","last_became_peered":"2024-01-19T04:55:19.496452+0000","last_unstale":"2024-01-19T04:55:20.492909+0000","last_undegraded":"2024-01-19T04:55:20.492909+0000","last_fullsized":"2024-01-19T04:55:20.492909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491153+0000","last_change":"2024-01-19T04:55:18.529231+0000","last_active":"2024-01-19T04:55:20.491153+0000","last_peered":"2024-01-19T04:55:20.491153+0000","last_clean":"2024-01-19T04:55:20.491153+0000","last_became_active":"2024-01-19T04:55:18.528957+0000","last_became_peered":"2024-01-19T04:55:18.528957+0000","last_unstale":"2024-01-19T04:55:20.491153+0000","last_undegraded":"2024-01-19T04:55:20.491153+0000","last_fullsized":"2024-01-19T04:55:20.491153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973545+0000","last_change":"2024-01-19T04:55:19.504814+0000","last_active":"2024-01-19T04:55:20.973545+0000","last_peered":"2024-01-19T04:55:20.973545+0000","last_clean":"2024-01-19T04:55:20.973545+0000","last_became_active":"2024-01-19T04:55:19.503588+0000","last_became_peered":"2024-01-19T04:55:19.503588+0000","last_unstale":"2024-01-19T04:55:20.973545+0000","last_undegraded":"2024-01-19T04:55:20.973545+0000","last_fullsized":"2024-01-19T04:55:20.973545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491129+0000","last_change":"2024-01-19T04:55:18.526393+0000","last_active":"2024-01-19T04:55:20.491129+0000","last_peered":"2024-01-19T04:55:20.491129+0000","last_clean":"2024-01-19T04:55:20.491129+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.491129+0000","last_undegraded":"2024-01-19T04:55:20.491129+0000","last_fullsized":"2024-01-19T04:55:20.491129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973518+0000","last_change":"2024-01-19T04:55:19.507971+0000","last_active":"2024-01-19T04:55:20.973518+0000","last_peered":"2024-01-19T04:55:20.973518+0000","last_clean":"2024-01-19T04:55:20.973518+0000","last_became_active":"2024-01-19T04:55:19.507303+0000","last_became_peered":"2024-01-19T04:55:19.507303+0000","last_unstale":"2024-01-19T04:55:20.973518+0000","last_undegraded":"2024-01-19T04:55:20.973518+0000","last_fullsized":"2024-01-19T04:55:20.973518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671371+0000","last_change":"2024-01-19T04:55:18.532050+0000","last_active":"2024-01-19T04:55:20.671371+0000","last_peered":"2024-01-19T04:55:20.671371+0000","last_clean":"2024-01-19T04:55:20.671371+0000","last_became_active":"2024-01-19T04:55:18.531639+0000","last_became_peered":"2024-01-19T04:55:18.531639+0000","last_unstale":"2024-01-19T04:55:20.671371+0000","last_undegraded":"2024-01-19T04:55:20.671371+0000","last_fullsized":"2024-01-19T04:55:20.671371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:17:18.531824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973276+0000","last_change":"2024-01-19T04:55:19.504174+0000","last_active":"2024-01-19T04:55:20.973276+0000","last_peered":"2024-01-19T04:55:20.973276+0000","last_clean":"2024-01-19T04:55:20.973276+0000","last_became_active":"2024-01-19T04:55:19.503547+0000","last_became_peered":"2024-01-19T04:55:19.503547+0000","last_unstale":"2024-01-19T04:55:20.973276+0000","last_undegraded":"2024-01-19T04:55:20.973276+0000","last_fullsized":"2024-01-19T04:55:20.973276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492450+0000","last_change":"2024-01-19T04:55:19.499280+0000","last_active":"2024-01-19T04:55:20.492450+0000","last_peered":"2024-01-19T04:55:20.492450+0000","last_clean":"2024-01-19T04:55:20.492450+0000","last_became_active":"2024-01-19T04:55:19.499013+0000","last_became_peered":"2024-01-19T04:55:19.499013+0000","last_unstale":"2024-01-19T04:55:20.492450+0000","last_undegraded":"2024-01-19T04:55:20.492450+0000","last_fullsized":"2024-01-19T04:55:20.492450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671111+0000","last_change":"2024-01-19T04:55:18.529039+0000","last_active":"2024-01-19T04:55:20.671111+0000","last_peered":"2024-01-19T04:55:20.671111+0000","last_clean":"2024-01-19T04:55:20.671111+0000","last_became_active":"2024-01-19T04:55:18.528779+0000","last_became_peered":"2024-01-19T04:55:18.528779+0000","last_unstale":"2024-01-19T04:55:20.671111+0000","last_undegraded":"2024-01-19T04:55:20.671111+0000","last_fullsized":"2024-01-19T04:55:20.671111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:18:53.418269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492561+0000","last_change":"2024-01-19T04:55:19.499057+0000","last_active":"2024-01-19T04:55:20.492561+0000","last_peered":"2024-01-19T04:55:20.492561+0000","last_clean":"2024-01-19T04:55:20.492561+0000","last_became_active":"2024-01-19T04:55:19.498786+0000","last_became_peered":"2024-01-19T04:55:19.498786+0000","last_unstale":"2024-01-19T04:55:20.492561+0000","last_undegraded":"2024-01-19T04:55:20.492561+0000","last_fullsized":"2024-01-19T04:55:20.492561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671370+0000","last_change":"2024-01-19T04:55:18.532007+0000","last_active":"2024-01-19T04:55:20.671370+0000","last_peered":"2024-01-19T04:55:20.671370+0000","last_clean":"2024-01-19T04:55:20.671370+0000","last_became_active":"2024-01-19T04:55:18.531693+0000","last_became_peered":"2024-01-19T04:55:18.531693+0000","last_unstale":"2024-01-19T04:55:20.671370+0000","last_undegraded":"2024-01-19T04:55:20.671370+0000","last_fullsized":"2024-01-19T04:55:20.671370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825366+0000","last_change":"2024-01-19T04:55:19.500839+0000","last_active":"2024-01-19T04:55:20.825366+0000","last_peered":"2024-01-19T04:55:20.825366+0000","last_clean":"2024-01-19T04:55:20.825366+0000","last_became_active":"2024-01-19T04:55:19.500206+0000","last_became_peered":"2024-01-19T04:55:19.500206+0000","last_unstale":"2024-01-19T04:55:20.825366+0000","last_undegraded":"2024-01-19T04:55:20.825366+0000","last_fullsized":"2024-01-19T04:55:20.825366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973811+0000","last_change":"2024-01-19T04:55:18.532935+0000","last_active":"2024-01-19T04:55:20.973811+0000","last_peered":"2024-01-19T04:55:20.973811+0000","last_clean":"2024-01-19T04:55:20.973811+0000","last_became_active":"2024-01-19T04:55:18.532664+0000","last_became_peered":"2024-01-19T04:55:18.532664+0000","last_unstale":"2024-01-19T04:55:20.973811+0000","last_undegraded":"2024-01-19T04:55:20.973811+0000","last_fullsized":"2024-01-19T04:55:20.973811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973712+0000","last_change":"2024-01-19T04:55:19.508271+0000","last_active":"2024-01-19T04:55:20.973712+0000","last_peered":"2024-01-19T04:55:20.973712+0000","last_clean":"2024-01-19T04:55:20.973712+0000","last_became_active":"2024-01-19T04:55:19.507496+0000","last_became_peered":"2024-01-19T04:55:19.507496+0000","last_unstale":"2024-01-19T04:55:20.973712+0000","last_undegraded":"2024-01-19T04:55:20.973712+0000","last_fullsized":"2024-01-19T04:55:20.973712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825450+0000","last_change":"2024-01-19T04:55:18.531059+0000","last_active":"2024-01-19T04:55:20.825450+0000","last_peered":"2024-01-19T04:55:20.825450+0000","last_clean":"2024-01-19T04:55:20.825450+0000","last_became_active":"2024-01-19T04:55:18.530752+0000","last_became_peered":"2024-01-19T04:55:18.530752+0000","last_unstale":"2024-01-19T04:55:20.825450+0000","last_undegraded":"2024-01-19T04:55:20.825450+0000","last_fullsized":"2024-01-19T04:55:20.825450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:04:18.004474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:55:31.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:55:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.442+0000 7f6fb4db6640 1 Processor -- start 2024-01-19T04:55:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.442+0000 7f6fb4db6640 1 -- start start 2024-01-19T04:55:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.443+0000 7f6fb4db6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006e8d0 0x7f6fb00d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.443+0000 7f6fb4db6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fb006ecb0 0x7f6fb00cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.443+0000 7f6fb4db6640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb00d0040 con 0x7f6fb00cee00 2024-01-19T04:55:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.443+0000 7f6fb4db6640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb00cd500 con 0x7f6fb006ecb0 2024-01-19T04:55:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.443+0000 7f6fb4db6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb00cb800 con 0x7f6fb006e8d0 2024-01-19T04:55:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.443+0000 7f6faed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fb006ecb0 0x7f6fb00cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6faed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fb006ecb0 0x7f6fb00cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43128/0 (socket says 172.21.15.31:43128) 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6faed9d640 1 -- 172.21.15.31:0/2115187887 learned_addr learned my addr 172.21.15.31:0/2115187887 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6fae59c640 1 --2- 172.21.15.31:0/2115187887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006e8d0 0x7f6fb00d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6faed9d640 1 -- 172.21.15.31:0/2115187887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006e8d0 msgr2=0x7f6fb00d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6faed9d640 1 --2- 172.21.15.31:0/2115187887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006e8d0 0x7f6fb00d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6faed9d640 1 -- 172.21.15.31:0/2115187887 >> v1:172.21.15.137:6789/0 conn(0x7f6fb00cee00 legacy=0x7f6fb015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6faed9d640 1 -- 172.21.15.31:0/2115187887 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb00cbae0 con 0x7f6fb006ecb0 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.444+0000 7f6faed9d640 1 --2- 172.21.15.31:0/2115187887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006ecb0 0x7f6fb00cd9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009870 tx=0x7f6fa802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc6eb1c5bd04d61 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fad59a640 1 -- 172.21.15.31:0/2115187887 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80317c0 con 0x7f6fb006ecb0 2024-01-19T04:55:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fad59a640 1 -- 172.21.15.31:0/2115187887 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8031920 con 0x7f6fb006ecb0 2024-01-19T04:55:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fad59a640 1 -- 172.21.15.31:0/2115187887 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8031cb0 con 0x7f6fb006ecb0 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 -- 172.21.15.31:0/2115187887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006ecb0 msgr2=0x7f6fb00cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/2115187887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006ecb0 0x7f6fb00cd9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009870 tx=0x7f6fa802b860 comp rx=0 tx=0).stop 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 -- 172.21.15.31:0/2115187887 shutdown_connections 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/2115187887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006ecb0 0x7f6fb00cd9e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/2115187887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006e8d0 0x7f6fb00d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 -- 172.21.15.31:0/2115187887 >> 172.21.15.31:0/2115187887 conn(0x7f6fb00b2c60 msgr2=0x7f6fb00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 -- 172.21.15.31:0/2115187887 shutdown_connections 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.445+0000 7f6fb4db6640 1 -- 172.21.15.31:0/2115187887 wait complete. 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 Processor -- start 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 -- start start 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006e8d0 0x7f6fb00c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006ecb0 0x7f6fb0151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb00cee00 0x7f6fb0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb00c81e0 con 0x7f6fb006e8d0 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb00c54e0 con 0x7f6fb00cee00 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fb4db6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb00c57c0 con 0x7f6fb006ecb0 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006e8d0 0x7f6fb00c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006e8d0 0x7f6fb00c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43134/0 (socket says 172.21.15.31:43134) 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.447+0000 7f6fae59c640 1 -- 172.21.15.31:0/3420620560 learned_addr learned my addr 172.21.15.31:0/3420620560 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6faed9d640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb00cee00 0x7f6fb0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fadd9b640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006ecb0 0x7f6fb0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fae59c640 1 -- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006ecb0 msgr2=0x7f6fb0151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fae59c640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006ecb0 0x7f6fb0151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fae59c640 1 -- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb00cee00 msgr2=0x7f6fb0151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fae59c640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb00cee00 0x7f6fb0151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fae59c640 1 -- 172.21.15.31:0/3420620560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80094d0 con 0x7f6fb006e8d0 2024-01-19T04:55:31.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fae59c640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006e8d0 0x7f6fb00c4c40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000ccd0 tx=0x7f6fa000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/3420620560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0003990 con 0x7f6fb006e8d0 2024-01-19T04:55:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/3420620560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0003af0 con 0x7f6fb006e8d0 2024-01-19T04:55:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb00c5aa0 con 0x7f6fb006e8d0 2024-01-19T04:55:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/3420620560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0004020 con 0x7f6fb006e8d0 2024-01-19T04:55:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.448+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb00c5180 con 0x7f6fb006e8d0 2024-01-19T04:55:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.449+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb0074a80 con 0x7f6fb006e8d0 2024-01-19T04:55:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.449+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/3420620560 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6fa0004180 con 0x7f6fb006e8d0 2024-01-19T04:55:31.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.450+0000 7f6fa6ffd640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f9006aed0 0x7f6f9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.450+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/3420620560 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fa0089a00 con 0x7f6fb006e8d0 2024-01-19T04:55:31.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.450+0000 7f6fadd9b640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f9006aed0 0x7f6f9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.451+0000 7f6fadd9b640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f9006aed0 0x7f6f9006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6fb01588c0 tx=0x7f6f98009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.454+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/3420620560 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6fa00548d0 con 0x7f6fb006e8d0 2024-01-19T04:55:31.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.631+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6fb00cf260 con 0x7f6f9006aed0 2024-01-19T04:55:31.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.639+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/3420620560 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264621 (secure 0 0 0) 0x7f6fb00cf260 con 0x7f6f9006aed0 2024-01-19T04:55:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:55:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f9006aed0 msgr2=0x7f6f9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f9006aed0 0x7f6f9006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6fb01588c0 tx=0x7f6f98009040 comp rx=0 tx=0).stop 2024-01-19T04:55:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006e8d0 msgr2=0x7f6fb00c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006e8d0 0x7f6fb00c4c40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000ccd0 tx=0x7f6fa000a5c0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 shutdown_connections 2024-01-19T04:55:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6f9006aed0 0x7f6f9006d390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb00cee00 0x7f6fb0151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6fb006ecb0 0x7f6fb0151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 --2- 172.21.15.31:0/3420620560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb006e8d0 0x7f6fb00c4c40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 >> 172.21.15.31:0/3420620560 conn(0x7f6fb00b2c60 msgr2=0x7f6fb0159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 shutdown_connections 2024-01-19T04:55:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.642+0000 7f6fb4db6640 1 -- 172.21.15.31:0/3420620560 wait complete. 2024-01-19T04:55:31.652 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-01-19T04:55:31.564449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219036,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.074461"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671430+0000","last_change":"2024-01-19T04:55:19.508084+0000","last_active":"2024-01-19T04:55:20.671430+0000","last_peered":"2024-01-19T04:55:20.671430+0000","last_clean":"2024-01-19T04:55:20.671430+0000","last_became_active":"2024-01-19T04:55:19.507540+0000","last_became_peered":"2024-01-19T04:55:19.507540+0000","last_unstale":"2024-01-19T04:55:20.671430+0000","last_undegraded":"2024-01-19T04:55:20.671430+0000","last_fullsized":"2024-01-19T04:55:20.671430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:05.071465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491274+0000","last_change":"2024-01-19T04:55:19.503341+0000","last_active":"2024-01-19T04:55:20.491274+0000","last_peered":"2024-01-19T04:55:20.491274+0000","last_clean":"2024-01-19T04:55:20.491274+0000","last_became_active":"2024-01-19T04:55:19.503039+0000","last_became_peered":"2024-01-19T04:55:19.503039+0000","last_unstale":"2024-01-19T04:55:20.491274+0000","last_undegraded":"2024-01-19T04:55:20.491274+0000","last_fullsized":"2024-01-19T04:55:20.491274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825350+0000","last_change":"2024-01-19T04:55:19.498911+0000","last_active":"2024-01-19T04:55:20.825350+0000","last_peered":"2024-01-19T04:55:20.825350+0000","last_clean":"2024-01-19T04:55:20.825350+0000","last_became_active":"2024-01-19T04:55:19.498070+0000","last_became_peered":"2024-01-19T04:55:19.498070+0000","last_unstale":"2024-01-19T04:55:20.825350+0000","last_undegraded":"2024-01-19T04:55:20.825350+0000","last_fullsized":"2024-01-19T04:55:20.825350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492501+0000","last_change":"2024-01-19T04:55:19.501815+0000","last_active":"2024-01-19T04:55:20.492501+0000","last_peered":"2024-01-19T04:55:20.492501+0000","last_clean":"2024-01-19T04:55:20.492501+0000","last_became_active":"2024-01-19T04:55:19.501554+0000","last_became_peered":"2024-01-19T04:55:19.501554+0000","last_unstale":"2024-01-19T04:55:20.492501+0000","last_undegraded":"2024-01-19T04:55:20.492501+0000","last_fullsized":"2024-01-19T04:55:20.492501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508864+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.507962+0000","last_became_peered":"2024-01-19T04:55:19.507962+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492193+0000","last_change":"2024-01-19T04:55:19.504292+0000","last_active":"2024-01-19T04:55:20.492193+0000","last_peered":"2024-01-19T04:55:20.492193+0000","last_clean":"2024-01-19T04:55:20.492193+0000","last_became_active":"2024-01-19T04:55:19.504021+0000","last_became_peered":"2024-01-19T04:55:19.504021+0000","last_unstale":"2024-01-19T04:55:20.492193+0000","last_undegraded":"2024-01-19T04:55:20.492193+0000","last_fullsized":"2024-01-19T04:55:20.492193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670963+0000","last_change":"2024-01-19T04:55:19.507244+0000","last_active":"2024-01-19T04:55:20.670963+0000","last_peered":"2024-01-19T04:55:20.670963+0000","last_clean":"2024-01-19T04:55:20.670963+0000","last_became_active":"2024-01-19T04:55:19.506995+0000","last_became_peered":"2024-01-19T04:55:19.506995+0000","last_unstale":"2024-01-19T04:55:20.670963+0000","last_undegraded":"2024-01-19T04:55:20.670963+0000","last_fullsized":"2024-01-19T04:55:20.670963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.489334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973302+0000","last_change":"2024-01-19T04:55:19.505311+0000","last_active":"2024-01-19T04:55:20.973302+0000","last_peered":"2024-01-19T04:55:20.973302+0000","last_clean":"2024-01-19T04:55:20.973302+0000","last_became_active":"2024-01-19T04:55:19.504534+0000","last_became_peered":"2024-01-19T04:55:19.504534+0000","last_unstale":"2024-01-19T04:55:20.973302+0000","last_undegraded":"2024-01-19T04:55:20.973302+0000","last_fullsized":"2024-01-19T04:55:20.973302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671451+0000","last_change":"2024-01-19T04:55:19.508793+0000","last_active":"2024-01-19T04:55:20.671451+0000","last_peered":"2024-01-19T04:55:20.671451+0000","last_clean":"2024-01-19T04:55:20.671451+0000","last_became_active":"2024-01-19T04:55:19.508267+0000","last_became_peered":"2024-01-19T04:55:19.508267+0000","last_unstale":"2024-01-19T04:55:20.671451+0000","last_undegraded":"2024-01-19T04:55:20.671451+0000","last_fullsized":"2024-01-19T04:55:20.671451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:49:04.554504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973577+0000","last_change":"2024-01-19T04:55:19.506536+0000","last_active":"2024-01-19T04:55:20.973577+0000","last_peered":"2024-01-19T04:55:20.973577+0000","last_clean":"2024-01-19T04:55:20.973577+0000","last_became_active":"2024-01-19T04:55:19.506239+0000","last_became_peered":"2024-01-19T04:55:19.506239+0000","last_unstale":"2024-01-19T04:55:20.973577+0000","last_undegraded":"2024-01-19T04:55:20.973577+0000","last_fullsized":"2024-01-19T04:55:20.973577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973471+0000","last_change":"2024-01-19T04:55:19.502406+0000","last_active":"2024-01-19T04:55:20.973471+0000","last_peered":"2024-01-19T04:55:20.973471+0000","last_clean":"2024-01-19T04:55:20.973471+0000","last_became_active":"2024-01-19T04:55:19.502144+0000","last_became_peered":"2024-01-19T04:55:19.502144+0000","last_unstale":"2024-01-19T04:55:20.973471+0000","last_undegraded":"2024-01-19T04:55:20.973471+0000","last_fullsized":"2024-01-19T04:55:20.973471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493066+0000","last_change":"2024-01-19T04:55:19.503258+0000","last_active":"2024-01-19T04:55:20.493066+0000","last_peered":"2024-01-19T04:55:20.493066+0000","last_clean":"2024-01-19T04:55:20.493066+0000","last_became_active":"2024-01-19T04:55:19.503006+0000","last_became_peered":"2024-01-19T04:55:19.503006+0000","last_unstale":"2024-01-19T04:55:20.493066+0000","last_undegraded":"2024-01-19T04:55:20.493066+0000","last_fullsized":"2024-01-19T04:55:20.493066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973473+0000","last_change":"2024-01-19T04:55:19.507886+0000","last_active":"2024-01-19T04:55:20.973473+0000","last_peered":"2024-01-19T04:55:20.973473+0000","last_clean":"2024-01-19T04:55:20.973473+0000","last_became_active":"2024-01-19T04:55:19.507443+0000","last_became_peered":"2024-01-19T04:55:19.507443+0000","last_unstale":"2024-01-19T04:55:20.973473+0000","last_undegraded":"2024-01-19T04:55:20.973473+0000","last_fullsized":"2024-01-19T04:55:20.973473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825087+0000","last_change":"2024-01-19T04:55:19.502972+0000","last_active":"2024-01-19T04:55:20.825087+0000","last_peered":"2024-01-19T04:55:20.825087+0000","last_clean":"2024-01-19T04:55:20.825087+0000","last_became_active":"2024-01-19T04:55:19.502612+0000","last_became_peered":"2024-01-19T04:55:19.502612+0000","last_unstale":"2024-01-19T04:55:20.825087+0000","last_undegraded":"2024-01-19T04:55:20.825087+0000","last_fullsized":"2024-01-19T04:55:20.825087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:13.926376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491125+0000","last_change":"2024-01-19T04:55:19.503794+0000","last_active":"2024-01-19T04:55:20.491125+0000","last_peered":"2024-01-19T04:55:20.491125+0000","last_clean":"2024-01-19T04:55:20.491125+0000","last_became_active":"2024-01-19T04:55:19.503516+0000","last_became_peered":"2024-01-19T04:55:19.503516+0000","last_unstale":"2024-01-19T04:55:20.491125+0000","last_undegraded":"2024-01-19T04:55:20.491125+0000","last_fullsized":"2024-01-19T04:55:20.491125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491667+0000","last_change":"2024-01-19T04:55:19.500410+0000","last_active":"2024-01-19T04:55:20.491667+0000","last_peered":"2024-01-19T04:55:20.491667+0000","last_clean":"2024-01-19T04:55:20.491667+0000","last_became_active":"2024-01-19T04:55:19.500147+0000","last_became_peered":"2024-01-19T04:55:19.500147+0000","last_unstale":"2024-01-19T04:55:20.491667+0000","last_undegraded":"2024-01-19T04:55:20.491667+0000","last_fullsized":"2024-01-19T04:55:20.491667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493061+0000","last_change":"2024-01-19T04:55:19.507266+0000","last_active":"2024-01-19T04:55:20.493061+0000","last_peered":"2024-01-19T04:55:20.493061+0000","last_clean":"2024-01-19T04:55:20.493061+0000","last_became_active":"2024-01-19T04:55:19.506970+0000","last_became_peered":"2024-01-19T04:55:19.506970+0000","last_unstale":"2024-01-19T04:55:20.493061+0000","last_undegraded":"2024-01-19T04:55:20.493061+0000","last_fullsized":"2024-01-19T04:55:20.493061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973133+0000","last_change":"2024-01-19T04:55:19.501787+0000","last_active":"2024-01-19T04:55:20.973133+0000","last_peered":"2024-01-19T04:55:20.973133+0000","last_clean":"2024-01-19T04:55:20.973133+0000","last_became_active":"2024-01-19T04:55:19.501389+0000","last_became_peered":"2024-01-19T04:55:19.501389+0000","last_unstale":"2024-01-19T04:55:20.973133+0000","last_undegraded":"2024-01-19T04:55:20.973133+0000","last_fullsized":"2024-01-19T04:55:20.973133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:23.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671369+0000","last_change":"2024-01-19T04:55:19.507131+0000","last_active":"2024-01-19T04:55:20.671369+0000","last_peered":"2024-01-19T04:55:20.671369+0000","last_clean":"2024-01-19T04:55:20.671369+0000","last_became_active":"2024-01-19T04:55:19.506871+0000","last_became_peered":"2024-01-19T04:55:19.506871+0000","last_unstale":"2024-01-19T04:55:20.671369+0000","last_undegraded":"2024-01-19T04:55:20.671369+0000","last_fullsized":"2024-01-19T04:55:20.671369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:29:51.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973517+0000","last_change":"2024-01-19T04:55:19.504339+0000","last_active":"2024-01-19T04:55:20.973517+0000","last_peered":"2024-01-19T04:55:20.973517+0000","last_clean":"2024-01-19T04:55:20.973517+0000","last_became_active":"2024-01-19T04:55:19.504087+0000","last_became_peered":"2024-01-19T04:55:19.504087+0000","last_unstale":"2024-01-19T04:55:20.973517+0000","last_undegraded":"2024-01-19T04:55:20.973517+0000","last_fullsized":"2024-01-19T04:55:20.973517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825223+0000","last_change":"2024-01-19T04:55:19.510243+0000","last_active":"2024-01-19T04:55:20.825223+0000","last_peered":"2024-01-19T04:55:20.825223+0000","last_clean":"2024-01-19T04:55:20.825223+0000","last_became_active":"2024-01-19T04:55:19.509971+0000","last_became_peered":"2024-01-19T04:55:19.509971+0000","last_unstale":"2024-01-19T04:55:20.825223+0000","last_undegraded":"2024-01-19T04:55:20.825223+0000","last_fullsized":"2024-01-19T04:55:20.825223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492345+0000","last_change":"2024-01-19T04:55:19.509452+0000","last_active":"2024-01-19T04:55:20.492345+0000","last_peered":"2024-01-19T04:55:20.492345+0000","last_clean":"2024-01-19T04:55:20.492345+0000","last_became_active":"2024-01-19T04:55:19.507159+0000","last_became_peered":"2024-01-19T04:55:19.507159+0000","last_unstale":"2024-01-19T04:55:20.492345+0000","last_undegraded":"2024-01-19T04:55:20.492345+0000","last_fullsized":"2024-01-19T04:55:20.492345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973182+0000","last_change":"2024-01-19T04:55:19.508179+0000","last_active":"2024-01-19T04:55:20.973182+0000","last_peered":"2024-01-19T04:55:20.973182+0000","last_clean":"2024-01-19T04:55:20.973182+0000","last_became_active":"2024-01-19T04:55:19.507867+0000","last_became_peered":"2024-01-19T04:55:19.507867+0000","last_unstale":"2024-01-19T04:55:20.973182+0000","last_undegraded":"2024-01-19T04:55:20.973182+0000","last_fullsized":"2024-01-19T04:55:20.973182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491882+0000","last_change":"2024-01-19T04:55:19.500977+0000","last_active":"2024-01-19T04:55:20.491882+0000","last_peered":"2024-01-19T04:55:20.491882+0000","last_clean":"2024-01-19T04:55:20.491882+0000","last_became_active":"2024-01-19T04:55:19.500722+0000","last_became_peered":"2024-01-19T04:55:19.500722+0000","last_unstale":"2024-01-19T04:55:20.491882+0000","last_undegraded":"2024-01-19T04:55:20.491882+0000","last_fullsized":"2024-01-19T04:55:20.491882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973622+0000","last_change":"2024-01-19T04:55:19.505026+0000","last_active":"2024-01-19T04:55:20.973622+0000","last_peered":"2024-01-19T04:55:20.973622+0000","last_clean":"2024-01-19T04:55:20.973622+0000","last_became_active":"2024-01-19T04:55:19.504732+0000","last_became_peered":"2024-01-19T04:55:19.504732+0000","last_unstale":"2024-01-19T04:55:20.973622+0000","last_undegraded":"2024-01-19T04:55:20.973622+0000","last_fullsized":"2024-01-19T04:55:20.973622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973617+0000","last_change":"2024-01-19T04:55:19.508736+0000","last_active":"2024-01-19T04:55:20.973617+0000","last_peered":"2024-01-19T04:55:20.973617+0000","last_clean":"2024-01-19T04:55:20.973617+0000","last_became_active":"2024-01-19T04:55:19.508432+0000","last_became_peered":"2024-01-19T04:55:19.508432+0000","last_unstale":"2024-01-19T04:55:20.973617+0000","last_undegraded":"2024-01-19T04:55:20.973617+0000","last_fullsized":"2024-01-19T04:55:20.973617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491221+0000","last_change":"2024-01-19T04:55:19.508491+0000","last_active":"2024-01-19T04:55:20.491221+0000","last_peered":"2024-01-19T04:55:20.491221+0000","last_clean":"2024-01-19T04:55:20.491221+0000","last_became_active":"2024-01-19T04:55:19.508174+0000","last_became_peered":"2024-01-19T04:55:19.508174+0000","last_unstale":"2024-01-19T04:55:20.491221+0000","last_undegraded":"2024-01-19T04:55:20.491221+0000","last_fullsized":"2024-01-19T04:55:20.491221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973603+0000","last_change":"2024-01-19T04:55:19.506159+0000","last_active":"2024-01-19T04:55:20.973603+0000","last_peered":"2024-01-19T04:55:20.973603+0000","last_clean":"2024-01-19T04:55:20.973603+0000","last_became_active":"2024-01-19T04:55:19.505854+0000","last_became_peered":"2024-01-19T04:55:19.505854+0000","last_unstale":"2024-01-19T04:55:20.973603+0000","last_undegraded":"2024-01-19T04:55:20.973603+0000","last_fullsized":"2024-01-19T04:55:20.973603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:51:32.707454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825356+0000","last_change":"2024-01-19T04:55:19.509241+0000","last_active":"2024-01-19T04:55:20.825356+0000","last_peered":"2024-01-19T04:55:20.825356+0000","last_clean":"2024-01-19T04:55:20.825356+0000","last_became_active":"2024-01-19T04:55:19.508899+0000","last_became_peered":"2024-01-19T04:55:19.508899+0000","last_unstale":"2024-01-19T04:55:20.825356+0000","last_undegraded":"2024-01-19T04:55:20.825356+0000","last_fullsized":"2024-01-19T04:55:20.825356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492316+0000","last_change":"2024-01-19T04:55:19.505518+0000","last_active":"2024-01-19T04:55:20.492316+0000","last_peered":"2024-01-19T04:55:20.492316+0000","last_clean":"2024-01-19T04:55:20.492316+0000","last_became_active":"2024-01-19T04:55:19.505203+0000","last_became_peered":"2024-01-19T04:55:19.505203+0000","last_unstale":"2024-01-19T04:55:20.492316+0000","last_undegraded":"2024-01-19T04:55:20.492316+0000","last_fullsized":"2024-01-19T04:55:20.492316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:24:07.263382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973044+0000","last_change":"2024-01-19T04:55:19.500467+0000","last_active":"2024-01-19T04:55:20.973044+0000","last_peered":"2024-01-19T04:55:20.973044+0000","last_clean":"2024-01-19T04:55:20.973044+0000","last_became_active":"2024-01-19T04:55:19.500146+0000","last_became_peered":"2024-01-19T04:55:19.500146+0000","last_unstale":"2024-01-19T04:55:20.973044+0000","last_undegraded":"2024-01-19T04:55:20.973044+0000","last_fullsized":"2024-01-19T04:55:20.973044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973209+0000","last_change":"2024-01-19T04:55:19.504033+0000","last_active":"2024-01-19T04:55:20.973209+0000","last_peered":"2024-01-19T04:55:20.973209+0000","last_clean":"2024-01-19T04:55:20.973209+0000","last_became_active":"2024-01-19T04:55:19.503555+0000","last_became_peered":"2024-01-19T04:55:19.503555+0000","last_unstale":"2024-01-19T04:55:20.973209+0000","last_undegraded":"2024-01-19T04:55:20.973209+0000","last_fullsized":"2024-01-19T04:55:20.973209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492127+0000","last_change":"2024-01-19T04:55:19.505336+0000","last_active":"2024-01-19T04:55:20.492127+0000","last_peered":"2024-01-19T04:55:20.492127+0000","last_clean":"2024-01-19T04:55:20.492127+0000","last_became_active":"2024-01-19T04:55:19.505035+0000","last_became_peered":"2024-01-19T04:55:19.505035+0000","last_unstale":"2024-01-19T04:55:20.492127+0000","last_undegraded":"2024-01-19T04:55:20.492127+0000","last_fullsized":"2024-01-19T04:55:20.492127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:38.193951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973572+0000","last_change":"2024-01-19T04:55:18.533807+0000","last_active":"2024-01-19T04:55:20.973572+0000","last_peered":"2024-01-19T04:55:20.973572+0000","last_clean":"2024-01-19T04:55:20.973572+0000","last_became_active":"2024-01-19T04:55:18.533426+0000","last_became_peered":"2024-01-19T04:55:18.533426+0000","last_unstale":"2024-01-19T04:55:20.973572+0000","last_undegraded":"2024-01-19T04:55:20.973572+0000","last_fullsized":"2024-01-19T04:55:20.973572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825363+0000","last_change":"2024-01-19T04:55:19.502475+0000","last_active":"2024-01-19T04:55:20.825363+0000","last_peered":"2024-01-19T04:55:20.825363+0000","last_clean":"2024-01-19T04:55:20.825363+0000","last_became_active":"2024-01-19T04:55:19.502036+0000","last_became_peered":"2024-01-19T04:55:19.502036+0000","last_unstale":"2024-01-19T04:55:20.825363+0000","last_undegraded":"2024-01-19T04:55:20.825363+0000","last_fullsized":"2024-01-19T04:55:20.825363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973270+0000","last_change":"2024-01-19T04:55:18.525394+0000","last_active":"2024-01-19T04:55:20.973270+0000","last_peered":"2024-01-19T04:55:20.973270+0000","last_clean":"2024-01-19T04:55:20.973270+0000","last_became_active":"2024-01-19T04:55:18.525118+0000","last_became_peered":"2024-01-19T04:55:18.525118+0000","last_unstale":"2024-01-19T04:55:20.973270+0000","last_undegraded":"2024-01-19T04:55:20.973270+0000","last_fullsized":"2024-01-19T04:55:20.973270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973314+0000","last_change":"2024-01-19T04:55:18.526820+0000","last_active":"2024-01-19T04:55:20.973314+0000","last_peered":"2024-01-19T04:55:20.973314+0000","last_clean":"2024-01-19T04:55:20.973314+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.973314+0000","last_undegraded":"2024-01-19T04:55:20.973314+0000","last_fullsized":"2024-01-19T04:55:20.973314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973308+0000","last_change":"2024-01-19T04:55:19.505074+0000","last_active":"2024-01-19T04:55:20.973308+0000","last_peered":"2024-01-19T04:55:20.973308+0000","last_clean":"2024-01-19T04:55:20.973308+0000","last_became_active":"2024-01-19T04:55:19.504093+0000","last_became_peered":"2024-01-19T04:55:19.504093+0000","last_unstale":"2024-01-19T04:55:20.973308+0000","last_undegraded":"2024-01-19T04:55:20.973308+0000","last_fullsized":"2024-01-19T04:55:20.973308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:25.102296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491874+0000","last_change":"2024-01-19T04:55:18.520981+0000","last_active":"2024-01-19T04:55:20.491874+0000","last_peered":"2024-01-19T04:55:20.491874+0000","last_clean":"2024-01-19T04:55:20.491874+0000","last_became_active":"2024-01-19T04:55:18.520110+0000","last_became_peered":"2024-01-19T04:55:18.520110+0000","last_unstale":"2024-01-19T04:55:20.491874+0000","last_undegraded":"2024-01-19T04:55:20.491874+0000","last_fullsized":"2024-01-19T04:55:20.491874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491068+0000","last_change":"2024-01-19T04:55:19.510473+0000","last_active":"2024-01-19T04:55:20.491068+0000","last_peered":"2024-01-19T04:55:20.491068+0000","last_clean":"2024-01-19T04:55:20.491068+0000","last_became_active":"2024-01-19T04:55:19.510207+0000","last_became_peered":"2024-01-19T04:55:19.510207+0000","last_unstale":"2024-01-19T04:55:20.491068+0000","last_undegraded":"2024-01-19T04:55:20.491068+0000","last_fullsized":"2024-01-19T04:55:20.491068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973574+0000","last_change":"2024-01-19T04:55:18.528905+0000","last_active":"2024-01-19T04:55:20.973574+0000","last_peered":"2024-01-19T04:55:20.973574+0000","last_clean":"2024-01-19T04:55:20.973574+0000","last_became_active":"2024-01-19T04:55:18.528580+0000","last_became_peered":"2024-01-19T04:55:18.528580+0000","last_unstale":"2024-01-19T04:55:20.973574+0000","last_undegraded":"2024-01-19T04:55:20.973574+0000","last_fullsized":"2024-01-19T04:55:20.973574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492708+0000","last_change":"2024-01-19T04:55:19.509724+0000","last_active":"2024-01-19T04:55:20.492708+0000","last_peered":"2024-01-19T04:55:20.492708+0000","last_clean":"2024-01-19T04:55:20.492708+0000","last_became_active":"2024-01-19T04:55:19.509431+0000","last_became_peered":"2024-01-19T04:55:19.509431+0000","last_unstale":"2024-01-19T04:55:20.492708+0000","last_undegraded":"2024-01-19T04:55:20.492708+0000","last_fullsized":"2024-01-19T04:55:20.492708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:21:09.868026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492870+0000","last_change":"2024-01-19T04:55:18.525788+0000","last_active":"2024-01-19T04:55:20.492870+0000","last_peered":"2024-01-19T04:55:20.492870+0000","last_clean":"2024-01-19T04:55:20.492870+0000","last_became_active":"2024-01-19T04:55:18.525521+0000","last_became_peered":"2024-01-19T04:55:18.525521+0000","last_unstale":"2024-01-19T04:55:20.492870+0000","last_undegraded":"2024-01-19T04:55:20.492870+0000","last_fullsized":"2024-01-19T04:55:20.492870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825497+0000","last_change":"2024-01-19T04:55:19.501054+0000","last_active":"2024-01-19T04:55:20.825497+0000","last_peered":"2024-01-19T04:55:20.825497+0000","last_clean":"2024-01-19T04:55:20.825497+0000","last_became_active":"2024-01-19T04:55:19.500559+0000","last_became_peered":"2024-01-19T04:55:19.500559+0000","last_unstale":"2024-01-19T04:55:20.825497+0000","last_undegraded":"2024-01-19T04:55:20.825497+0000","last_fullsized":"2024-01-19T04:55:20.825497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491237+0000","last_change":"2024-01-19T04:55:18.528505+0000","last_active":"2024-01-19T04:55:20.491237+0000","last_peered":"2024-01-19T04:55:20.491237+0000","last_clean":"2024-01-19T04:55:20.491237+0000","last_became_active":"2024-01-19T04:55:18.528221+0000","last_became_peered":"2024-01-19T04:55:18.528221+0000","last_unstale":"2024-01-19T04:55:20.491237+0000","last_undegraded":"2024-01-19T04:55:20.491237+0000","last_fullsized":"2024-01-19T04:55:20.491237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:28:22.050835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.493257+0000","last_change":"2024-01-19T04:55:19.507145+0000","last_active":"2024-01-19T04:55:20.493257+0000","last_peered":"2024-01-19T04:55:20.493257+0000","last_clean":"2024-01-19T04:55:20.493257+0000","last_became_active":"2024-01-19T04:55:19.506516+0000","last_became_peered":"2024-01-19T04:55:19.506516+0000","last_unstale":"2024-01-19T04:55:20.493257+0000","last_undegraded":"2024-01-19T04:55:20.493257+0000","last_fullsized":"2024-01-19T04:55:20.493257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:42.763778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973447+0000","last_change":"2024-01-19T04:55:18.527944+0000","last_active":"2024-01-19T04:55:20.973447+0000","last_peered":"2024-01-19T04:55:20.973447+0000","last_clean":"2024-01-19T04:55:20.973447+0000","last_became_active":"2024-01-19T04:55:18.527627+0000","last_became_peered":"2024-01-19T04:55:18.527627+0000","last_unstale":"2024-01-19T04:55:20.973447+0000","last_undegraded":"2024-01-19T04:55:20.973447+0000","last_fullsized":"2024-01-19T04:55:20.973447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671173+0000","last_change":"2024-01-19T04:55:19.508912+0000","last_active":"2024-01-19T04:55:20.671173+0000","last_peered":"2024-01-19T04:55:20.671173+0000","last_clean":"2024-01-19T04:55:20.671173+0000","last_became_active":"2024-01-19T04:55:19.508650+0000","last_became_peered":"2024-01-19T04:55:19.508650+0000","last_unstale":"2024-01-19T04:55:20.671173+0000","last_undegraded":"2024-01-19T04:55:20.671173+0000","last_fullsized":"2024-01-19T04:55:20.671173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:11:03.278269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825436+0000","last_change":"2024-01-19T04:55:18.529432+0000","last_active":"2024-01-19T04:55:20.825436+0000","last_peered":"2024-01-19T04:55:20.825436+0000","last_clean":"2024-01-19T04:55:20.825436+0000","last_became_active":"2024-01-19T04:55:18.529166+0000","last_became_peered":"2024-01-19T04:55:18.529166+0000","last_unstale":"2024-01-19T04:55:20.825436+0000","last_undegraded":"2024-01-19T04:55:20.825436+0000","last_fullsized":"2024-01-19T04:55:20.825436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825157+0000","last_change":"2024-01-19T04:55:19.507049+0000","last_active":"2024-01-19T04:55:20.825157+0000","last_peered":"2024-01-19T04:55:20.825157+0000","last_clean":"2024-01-19T04:55:20.825157+0000","last_became_active":"2024-01-19T04:55:19.506681+0000","last_became_peered":"2024-01-19T04:55:19.506681+0000","last_unstale":"2024-01-19T04:55:20.825157+0000","last_undegraded":"2024-01-19T04:55:20.825157+0000","last_fullsized":"2024-01-19T04:55:20.825157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:44:30.509690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492787+0000","last_change":"2024-01-19T04:55:18.522952+0000","last_active":"2024-01-19T04:55:20.492787+0000","last_peered":"2024-01-19T04:55:20.492787+0000","last_clean":"2024-01-19T04:55:20.492787+0000","last_became_active":"2024-01-19T04:55:18.522594+0000","last_became_peered":"2024-01-19T04:55:18.522594+0000","last_unstale":"2024-01-19T04:55:20.492787+0000","last_undegraded":"2024-01-19T04:55:20.492787+0000","last_fullsized":"2024-01-19T04:55:20.492787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671150+0000","last_change":"2024-01-19T04:55:19.509175+0000","last_active":"2024-01-19T04:55:20.671150+0000","last_peered":"2024-01-19T04:55:20.671150+0000","last_clean":"2024-01-19T04:55:20.671150+0000","last_became_active":"2024-01-19T04:55:19.508249+0000","last_became_peered":"2024-01-19T04:55:19.508249+0000","last_unstale":"2024-01-19T04:55:20.671150+0000","last_undegraded":"2024-01-19T04:55:20.671150+0000","last_fullsized":"2024-01-19T04:55:20.671150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973205+0000","last_change":"2024-01-19T04:55:18.525890+0000","last_active":"2024-01-19T04:55:20.973205+0000","last_peered":"2024-01-19T04:55:20.973205+0000","last_clean":"2024-01-19T04:55:20.973205+0000","last_became_active":"2024-01-19T04:55:18.525630+0000","last_became_peered":"2024-01-19T04:55:18.525630+0000","last_unstale":"2024-01-19T04:55:20.973205+0000","last_undegraded":"2024-01-19T04:55:20.973205+0000","last_fullsized":"2024-01-19T04:55:20.973205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973278+0000","last_change":"2024-01-19T04:55:19.503820+0000","last_active":"2024-01-19T04:55:20.973278+0000","last_peered":"2024-01-19T04:55:20.973278+0000","last_clean":"2024-01-19T04:55:20.973278+0000","last_became_active":"2024-01-19T04:55:19.503248+0000","last_became_peered":"2024-01-19T04:55:19.503248+0000","last_unstale":"2024-01-19T04:55:20.973278+0000","last_undegraded":"2024-01-19T04:55:20.973278+0000","last_fullsized":"2024-01-19T04:55:20.973278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:00:58.983208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973005+0000","last_change":"2024-01-19T04:55:18.521372+0000","last_active":"2024-01-19T04:55:20.973005+0000","last_peered":"2024-01-19T04:55:20.973005+0000","last_clean":"2024-01-19T04:55:20.973005+0000","last_became_active":"2024-01-19T04:55:18.521021+0000","last_became_peered":"2024-01-19T04:55:18.521021+0000","last_unstale":"2024-01-19T04:55:20.973005+0000","last_undegraded":"2024-01-19T04:55:20.973005+0000","last_fullsized":"2024-01-19T04:55:20.973005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671050+0000","last_change":"2024-01-19T04:55:19.507703+0000","last_active":"2024-01-19T04:55:20.671050+0000","last_peered":"2024-01-19T04:55:20.671050+0000","last_clean":"2024-01-19T04:55:20.671050+0000","last_became_active":"2024-01-19T04:55:19.507243+0000","last_became_peered":"2024-01-19T04:55:19.507243+0000","last_unstale":"2024-01-19T04:55:20.671050+0000","last_undegraded":"2024-01-19T04:55:20.671050+0000","last_fullsized":"2024-01-19T04:55:20.671050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492777+0000","last_change":"2024-01-19T04:55:18.531093+0000","last_active":"2024-01-19T04:55:20.492777+0000","last_peered":"2024-01-19T04:55:20.492777+0000","last_clean":"2024-01-19T04:55:20.492777+0000","last_became_active":"2024-01-19T04:55:18.530783+0000","last_became_peered":"2024-01-19T04:55:18.530783+0000","last_unstale":"2024-01-19T04:55:20.492777+0000","last_undegraded":"2024-01-19T04:55:20.492777+0000","last_fullsized":"2024-01-19T04:55:20.492777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:18:16.876671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671483+0000","last_change":"2024-01-19T04:55:19.506668+0000","last_active":"2024-01-19T04:55:20.671483+0000","last_peered":"2024-01-19T04:55:20.671483+0000","last_clean":"2024-01-19T04:55:20.671483+0000","last_became_active":"2024-01-19T04:55:19.506350+0000","last_became_peered":"2024-01-19T04:55:19.506350+0000","last_unstale":"2024-01-19T04:55:20.671483+0000","last_undegraded":"2024-01-19T04:55:20.671483+0000","last_fullsized":"2024-01-19T04:55:20.671483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:54:10.730504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973247+0000","last_change":"2024-01-19T04:55:18.523120+0000","last_active":"2024-01-19T04:55:20.973247+0000","last_peered":"2024-01-19T04:55:20.973247+0000","last_clean":"2024-01-19T04:55:20.973247+0000","last_became_active":"2024-01-19T04:55:18.522810+0000","last_became_peered":"2024-01-19T04:55:18.522810+0000","last_unstale":"2024-01-19T04:55:20.973247+0000","last_undegraded":"2024-01-19T04:55:20.973247+0000","last_fullsized":"2024-01-19T04:55:20.973247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:03:58.669737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671581+0000","last_change":"2024-01-19T04:55:19.508309+0000","last_active":"2024-01-19T04:55:20.671581+0000","last_peered":"2024-01-19T04:55:20.671581+0000","last_clean":"2024-01-19T04:55:20.671581+0000","last_became_active":"2024-01-19T04:55:19.507838+0000","last_became_peered":"2024-01-19T04:55:19.507838+0000","last_unstale":"2024-01-19T04:55:20.671581+0000","last_undegraded":"2024-01-19T04:55:20.671581+0000","last_fullsized":"2024-01-19T04:55:20.671581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:59:10.745652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491300+0000","last_change":"2024-01-19T04:55:18.526216+0000","last_active":"2024-01-19T04:55:20.491300+0000","last_peered":"2024-01-19T04:55:20.491300+0000","last_clean":"2024-01-19T04:55:20.491300+0000","last_became_active":"2024-01-19T04:55:18.524313+0000","last_became_peered":"2024-01-19T04:55:18.524313+0000","last_unstale":"2024-01-19T04:55:20.491300+0000","last_undegraded":"2024-01-19T04:55:20.491300+0000","last_fullsized":"2024-01-19T04:55:20.491300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492661+0000","last_change":"2024-01-19T04:55:19.501625+0000","last_active":"2024-01-19T04:55:20.492661+0000","last_peered":"2024-01-19T04:55:20.492661+0000","last_clean":"2024-01-19T04:55:20.492661+0000","last_became_active":"2024-01-19T04:55:19.500107+0000","last_became_peered":"2024-01-19T04:55:19.500107+0000","last_unstale":"2024-01-19T04:55:20.492661+0000","last_undegraded":"2024-01-19T04:55:20.492661+0000","last_fullsized":"2024-01-19T04:55:20.492661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:49:21.148045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973138+0000","last_change":"2024-01-19T04:55:18.525457+0000","last_active":"2024-01-19T04:55:20.973138+0000","last_peered":"2024-01-19T04:55:20.973138+0000","last_clean":"2024-01-19T04:55:20.973138+0000","last_became_active":"2024-01-19T04:55:18.525183+0000","last_became_peered":"2024-01-19T04:55:18.525183+0000","last_unstale":"2024-01-19T04:55:20.973138+0000","last_undegraded":"2024-01-19T04:55:20.973138+0000","last_fullsized":"2024-01-19T04:55:20.973138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:07:48.917251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671198+0000","last_change":"2024-01-19T04:55:19.508457+0000","last_active":"2024-01-19T04:55:20.671198+0000","last_peered":"2024-01-19T04:55:20.671198+0000","last_clean":"2024-01-19T04:55:20.671198+0000","last_became_active":"2024-01-19T04:55:19.506325+0000","last_became_peered":"2024-01-19T04:55:19.506325+0000","last_unstale":"2024-01-19T04:55:20.671198+0000","last_undegraded":"2024-01-19T04:55:20.671198+0000","last_fullsized":"2024-01-19T04:55:20.671198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:01:52.567337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-19T04:55:19.496795+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:55:19.496795+0000","last_peered":"2024-01-19T04:55:19.496795+0000","last_clean":"2024-01-19T04:55:19.496795+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:55:19.496795+0000","last_undegraded":"2024-01-19T04:55:19.496795+0000","last_fullsized":"2024-01-19T04:55:19.496795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492187+0000","last_change":"2024-01-19T04:55:18.520709+0000","last_active":"2024-01-19T04:55:20.492187+0000","last_peered":"2024-01-19T04:55:20.492187+0000","last_clean":"2024-01-19T04:55:20.492187+0000","last_became_active":"2024-01-19T04:55:18.520257+0000","last_became_peered":"2024-01-19T04:55:18.520257+0000","last_unstale":"2024-01-19T04:55:20.492187+0000","last_undegraded":"2024-01-19T04:55:20.492187+0000","last_fullsized":"2024-01-19T04:55:20.492187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973410+0000","last_change":"2024-01-19T04:55:19.507460+0000","last_active":"2024-01-19T04:55:20.973410+0000","last_peered":"2024-01-19T04:55:20.973410+0000","last_clean":"2024-01-19T04:55:20.973410+0000","last_became_active":"2024-01-19T04:55:19.507182+0000","last_became_peered":"2024-01-19T04:55:19.507182+0000","last_unstale":"2024-01-19T04:55:20.973410+0000","last_undegraded":"2024-01-19T04:55:20.973410+0000","last_fullsized":"2024-01-19T04:55:20.973410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670846+0000","last_change":"2024-01-19T04:55:18.531158+0000","last_active":"2024-01-19T04:55:20.670846+0000","last_peered":"2024-01-19T04:55:20.670846+0000","last_clean":"2024-01-19T04:55:20.670846+0000","last_became_active":"2024-01-19T04:55:18.530811+0000","last_became_peered":"2024-01-19T04:55:18.530811+0000","last_unstale":"2024-01-19T04:55:20.670846+0000","last_undegraded":"2024-01-19T04:55:20.670846+0000","last_fullsized":"2024-01-19T04:55:20.670846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:31.580264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825075+0000","last_change":"2024-01-19T04:55:19.503305+0000","last_active":"2024-01-19T04:55:20.825075+0000","last_peered":"2024-01-19T04:55:20.825075+0000","last_clean":"2024-01-19T04:55:20.825075+0000","last_became_active":"2024-01-19T04:55:19.502993+0000","last_became_peered":"2024-01-19T04:55:19.502993+0000","last_unstale":"2024-01-19T04:55:20.825075+0000","last_undegraded":"2024-01-19T04:55:20.825075+0000","last_fullsized":"2024-01-19T04:55:20.825075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:02:32.585555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973077+0000","last_change":"2024-01-19T04:55:18.523147+0000","last_active":"2024-01-19T04:55:20.973077+0000","last_peered":"2024-01-19T04:55:20.973077+0000","last_clean":"2024-01-19T04:55:20.973077+0000","last_became_active":"2024-01-19T04:55:18.522848+0000","last_became_peered":"2024-01-19T04:55:18.522848+0000","last_unstale":"2024-01-19T04:55:20.973077+0000","last_undegraded":"2024-01-19T04:55:20.973077+0000","last_fullsized":"2024-01-19T04:55:20.973077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491259+0000","last_change":"2024-01-19T04:55:19.500588+0000","last_active":"2024-01-19T04:55:20.491259+0000","last_peered":"2024-01-19T04:55:20.491259+0000","last_clean":"2024-01-19T04:55:20.491259+0000","last_became_active":"2024-01-19T04:55:19.500210+0000","last_became_peered":"2024-01-19T04:55:19.500210+0000","last_unstale":"2024-01-19T04:55:20.491259+0000","last_undegraded":"2024-01-19T04:55:20.491259+0000","last_fullsized":"2024-01-19T04:55:20.491259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:35.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492838+0000","last_change":"2024-01-19T04:55:18.521717+0000","last_active":"2024-01-19T04:55:20.492838+0000","last_peered":"2024-01-19T04:55:20.492838+0000","last_clean":"2024-01-19T04:55:20.492838+0000","last_became_active":"2024-01-19T04:55:18.521452+0000","last_became_peered":"2024-01-19T04:55:18.521452+0000","last_unstale":"2024-01-19T04:55:20.492838+0000","last_undegraded":"2024-01-19T04:55:20.492838+0000","last_fullsized":"2024-01-19T04:55:20.492838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492300+0000","last_change":"2024-01-19T04:55:19.497086+0000","last_active":"2024-01-19T04:55:20.492300+0000","last_peered":"2024-01-19T04:55:20.492300+0000","last_clean":"2024-01-19T04:55:20.492300+0000","last_became_active":"2024-01-19T04:55:19.496812+0000","last_became_peered":"2024-01-19T04:55:19.496812+0000","last_unstale":"2024-01-19T04:55:20.492300+0000","last_undegraded":"2024-01-19T04:55:20.492300+0000","last_fullsized":"2024-01-19T04:55:20.492300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:24.177458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491255+0000","last_change":"2024-01-19T04:55:18.531156+0000","last_active":"2024-01-19T04:55:20.491255+0000","last_peered":"2024-01-19T04:55:20.491255+0000","last_clean":"2024-01-19T04:55:20.491255+0000","last_became_active":"2024-01-19T04:55:18.530716+0000","last_became_peered":"2024-01-19T04:55:18.530716+0000","last_unstale":"2024-01-19T04:55:20.491255+0000","last_undegraded":"2024-01-19T04:55:20.491255+0000","last_fullsized":"2024-01-19T04:55:20.491255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:19:37.188659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492468+0000","last_change":"2024-01-19T04:55:19.497882+0000","last_active":"2024-01-19T04:55:20.492468+0000","last_peered":"2024-01-19T04:55:20.492468+0000","last_clean":"2024-01-19T04:55:20.492468+0000","last_became_active":"2024-01-19T04:55:19.497509+0000","last_became_peered":"2024-01-19T04:55:19.497509+0000","last_unstale":"2024-01-19T04:55:20.492468+0000","last_undegraded":"2024-01-19T04:55:20.492468+0000","last_fullsized":"2024-01-19T04:55:20.492468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671479+0000","last_change":"2024-01-19T04:55:18.539166+0000","last_active":"2024-01-19T04:55:20.671479+0000","last_peered":"2024-01-19T04:55:20.671479+0000","last_clean":"2024-01-19T04:55:20.671479+0000","last_became_active":"2024-01-19T04:55:18.538937+0000","last_became_peered":"2024-01-19T04:55:18.538937+0000","last_unstale":"2024-01-19T04:55:20.671479+0000","last_undegraded":"2024-01-19T04:55:20.671479+0000","last_fullsized":"2024-01-19T04:55:20.671479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:21:12.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491285+0000","last_change":"2024-01-19T04:55:19.499782+0000","last_active":"2024-01-19T04:55:20.491285+0000","last_peered":"2024-01-19T04:55:20.491285+0000","last_clean":"2024-01-19T04:55:20.491285+0000","last_became_active":"2024-01-19T04:55:19.499495+0000","last_became_peered":"2024-01-19T04:55:19.499495+0000","last_unstale":"2024-01-19T04:55:20.491285+0000","last_undegraded":"2024-01-19T04:55:20.491285+0000","last_fullsized":"2024-01-19T04:55:20.491285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:21.576294+0000","last_change":"2024-01-19T04:55:18.530049+0000","last_active":"2024-01-19T04:55:21.576294+0000","last_peered":"2024-01-19T04:55:21.576294+0000","last_clean":"2024-01-19T04:55:21.576294+0000","last_became_active":"2024-01-19T04:55:18.529767+0000","last_became_peered":"2024-01-19T04:55:18.529767+0000","last_unstale":"2024-01-19T04:55:21.576294+0000","last_undegraded":"2024-01-19T04:55:21.576294+0000","last_fullsized":"2024-01-19T04:55:21.576294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973714+0000","last_change":"2024-01-19T04:55:19.511086+0000","last_active":"2024-01-19T04:55:20.973714+0000","last_peered":"2024-01-19T04:55:20.973714+0000","last_clean":"2024-01-19T04:55:20.973714+0000","last_became_active":"2024-01-19T04:55:19.510781+0000","last_became_peered":"2024-01-19T04:55:19.510781+0000","last_unstale":"2024-01-19T04:55:20.973714+0000","last_undegraded":"2024-01-19T04:55:20.973714+0000","last_fullsized":"2024-01-19T04:55:20.973714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:08:17.861461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973635+0000","last_change":"2024-01-19T04:55:18.529646+0000","last_active":"2024-01-19T04:55:20.973635+0000","last_peered":"2024-01-19T04:55:20.973635+0000","last_clean":"2024-01-19T04:55:20.973635+0000","last_became_active":"2024-01-19T04:55:18.529403+0000","last_became_peered":"2024-01-19T04:55:18.529403+0000","last_unstale":"2024-01-19T04:55:20.973635+0000","last_undegraded":"2024-01-19T04:55:20.973635+0000","last_fullsized":"2024-01-19T04:55:20.973635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:08:23.172166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825397+0000","last_change":"2024-01-19T04:55:19.503772+0000","last_active":"2024-01-19T04:55:20.825397+0000","last_peered":"2024-01-19T04:55:20.825397+0000","last_clean":"2024-01-19T04:55:20.825397+0000","last_became_active":"2024-01-19T04:55:19.503188+0000","last_became_peered":"2024-01-19T04:55:19.503188+0000","last_unstale":"2024-01-19T04:55:20.825397+0000","last_undegraded":"2024-01-19T04:55:20.825397+0000","last_fullsized":"2024-01-19T04:55:20.825397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:23:39.979330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.670816+0000","last_change":"2024-01-19T04:55:18.536711+0000","last_active":"2024-01-19T04:55:20.670816+0000","last_peered":"2024-01-19T04:55:20.670816+0000","last_clean":"2024-01-19T04:55:20.670816+0000","last_became_active":"2024-01-19T04:55:18.536483+0000","last_became_peered":"2024-01-19T04:55:18.536483+0000","last_unstale":"2024-01-19T04:55:20.670816+0000","last_undegraded":"2024-01-19T04:55:20.670816+0000","last_fullsized":"2024-01-19T04:55:20.670816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:46:12.921084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492909+0000","last_change":"2024-01-19T04:55:19.496953+0000","last_active":"2024-01-19T04:55:20.492909+0000","last_peered":"2024-01-19T04:55:20.492909+0000","last_clean":"2024-01-19T04:55:20.492909+0000","last_became_active":"2024-01-19T04:55:19.496452+0000","last_became_peered":"2024-01-19T04:55:19.496452+0000","last_unstale":"2024-01-19T04:55:20.492909+0000","last_undegraded":"2024-01-19T04:55:20.492909+0000","last_fullsized":"2024-01-19T04:55:20.492909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:44.060195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491153+0000","last_change":"2024-01-19T04:55:18.529231+0000","last_active":"2024-01-19T04:55:20.491153+0000","last_peered":"2024-01-19T04:55:20.491153+0000","last_clean":"2024-01-19T04:55:20.491153+0000","last_became_active":"2024-01-19T04:55:18.528957+0000","last_became_peered":"2024-01-19T04:55:18.528957+0000","last_unstale":"2024-01-19T04:55:20.491153+0000","last_undegraded":"2024-01-19T04:55:20.491153+0000","last_fullsized":"2024-01-19T04:55:20.491153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:29.485126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973545+0000","last_change":"2024-01-19T04:55:19.504814+0000","last_active":"2024-01-19T04:55:20.973545+0000","last_peered":"2024-01-19T04:55:20.973545+0000","last_clean":"2024-01-19T04:55:20.973545+0000","last_became_active":"2024-01-19T04:55:19.503588+0000","last_became_peered":"2024-01-19T04:55:19.503588+0000","last_unstale":"2024-01-19T04:55:20.973545+0000","last_undegraded":"2024-01-19T04:55:20.973545+0000","last_fullsized":"2024-01-19T04:55:20.973545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.491129+0000","last_change":"2024-01-19T04:55:18.526393+0000","last_active":"2024-01-19T04:55:20.491129+0000","last_peered":"2024-01-19T04:55:20.491129+0000","last_clean":"2024-01-19T04:55:20.491129+0000","last_became_active":"2024-01-19T04:55:18.526076+0000","last_became_peered":"2024-01-19T04:55:18.526076+0000","last_unstale":"2024-01-19T04:55:20.491129+0000","last_undegraded":"2024-01-19T04:55:20.491129+0000","last_fullsized":"2024-01-19T04:55:20.491129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:36:43.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973518+0000","last_change":"2024-01-19T04:55:19.507971+0000","last_active":"2024-01-19T04:55:20.973518+0000","last_peered":"2024-01-19T04:55:20.973518+0000","last_clean":"2024-01-19T04:55:20.973518+0000","last_became_active":"2024-01-19T04:55:19.507303+0000","last_became_peered":"2024-01-19T04:55:19.507303+0000","last_unstale":"2024-01-19T04:55:20.973518+0000","last_undegraded":"2024-01-19T04:55:20.973518+0000","last_fullsized":"2024-01-19T04:55:20.973518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:28:35.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671371+0000","last_change":"2024-01-19T04:55:18.532050+0000","last_active":"2024-01-19T04:55:20.671371+0000","last_peered":"2024-01-19T04:55:20.671371+0000","last_clean":"2024-01-19T04:55:20.671371+0000","last_became_active":"2024-01-19T04:55:18.531639+0000","last_became_peered":"2024-01-19T04:55:18.531639+0000","last_unstale":"2024-01-19T04:55:20.671371+0000","last_undegraded":"2024-01-19T04:55:20.671371+0000","last_fullsized":"2024-01-19T04:55:20.671371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:17:18.531824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973276+0000","last_change":"2024-01-19T04:55:19.504174+0000","last_active":"2024-01-19T04:55:20.973276+0000","last_peered":"2024-01-19T04:55:20.973276+0000","last_clean":"2024-01-19T04:55:20.973276+0000","last_became_active":"2024-01-19T04:55:19.503547+0000","last_became_peered":"2024-01-19T04:55:19.503547+0000","last_unstale":"2024-01-19T04:55:20.973276+0000","last_undegraded":"2024-01-19T04:55:20.973276+0000","last_fullsized":"2024-01-19T04:55:20.973276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:08:12.939268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492450+0000","last_change":"2024-01-19T04:55:19.499280+0000","last_active":"2024-01-19T04:55:20.492450+0000","last_peered":"2024-01-19T04:55:20.492450+0000","last_clean":"2024-01-19T04:55:20.492450+0000","last_became_active":"2024-01-19T04:55:19.499013+0000","last_became_peered":"2024-01-19T04:55:19.499013+0000","last_unstale":"2024-01-19T04:55:20.492450+0000","last_undegraded":"2024-01-19T04:55:20.492450+0000","last_fullsized":"2024-01-19T04:55:20.492450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671111+0000","last_change":"2024-01-19T04:55:18.529039+0000","last_active":"2024-01-19T04:55:20.671111+0000","last_peered":"2024-01-19T04:55:20.671111+0000","last_clean":"2024-01-19T04:55:20.671111+0000","last_became_active":"2024-01-19T04:55:18.528779+0000","last_became_peered":"2024-01-19T04:55:18.528779+0000","last_unstale":"2024-01-19T04:55:20.671111+0000","last_undegraded":"2024-01-19T04:55:20.671111+0000","last_fullsized":"2024-01-19T04:55:20.671111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:18:53.418269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.492561+0000","last_change":"2024-01-19T04:55:19.499057+0000","last_active":"2024-01-19T04:55:20.492561+0000","last_peered":"2024-01-19T04:55:20.492561+0000","last_clean":"2024-01-19T04:55:20.492561+0000","last_became_active":"2024-01-19T04:55:19.498786+0000","last_became_peered":"2024-01-19T04:55:19.498786+0000","last_unstale":"2024-01-19T04:55:20.492561+0000","last_undegraded":"2024-01-19T04:55:20.492561+0000","last_fullsized":"2024-01-19T04:55:20.492561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:02:02.771775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.671370+0000","last_change":"2024-01-19T04:55:18.532007+0000","last_active":"2024-01-19T04:55:20.671370+0000","last_peered":"2024-01-19T04:55:20.671370+0000","last_clean":"2024-01-19T04:55:20.671370+0000","last_became_active":"2024-01-19T04:55:18.531693+0000","last_became_peered":"2024-01-19T04:55:18.531693+0000","last_unstale":"2024-01-19T04:55:20.671370+0000","last_undegraded":"2024-01-19T04:55:20.671370+0000","last_fullsized":"2024-01-19T04:55:20.671370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:51:41.758487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825366+0000","last_change":"2024-01-19T04:55:19.500839+0000","last_active":"2024-01-19T04:55:20.825366+0000","last_peered":"2024-01-19T04:55:20.825366+0000","last_clean":"2024-01-19T04:55:20.825366+0000","last_became_active":"2024-01-19T04:55:19.500206+0000","last_became_peered":"2024-01-19T04:55:19.500206+0000","last_unstale":"2024-01-19T04:55:20.825366+0000","last_undegraded":"2024-01-19T04:55:20.825366+0000","last_fullsized":"2024-01-19T04:55:20.825366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:40:32.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973811+0000","last_change":"2024-01-19T04:55:18.532935+0000","last_active":"2024-01-19T04:55:20.973811+0000","last_peered":"2024-01-19T04:55:20.973811+0000","last_clean":"2024-01-19T04:55:20.973811+0000","last_became_active":"2024-01-19T04:55:18.532664+0000","last_became_peered":"2024-01-19T04:55:18.532664+0000","last_unstale":"2024-01-19T04:55:20.973811+0000","last_undegraded":"2024-01-19T04:55:20.973811+0000","last_fullsized":"2024-01-19T04:55:20.973811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:30:19.291135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.973712+0000","last_change":"2024-01-19T04:55:19.508271+0000","last_active":"2024-01-19T04:55:20.973712+0000","last_peered":"2024-01-19T04:55:20.973712+0000","last_clean":"2024-01-19T04:55:20.973712+0000","last_became_active":"2024-01-19T04:55:19.507496+0000","last_became_peered":"2024-01-19T04:55:19.507496+0000","last_unstale":"2024-01-19T04:55:20.973712+0000","last_undegraded":"2024-01-19T04:55:20.973712+0000","last_fullsized":"2024-01-19T04:55:20.973712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:18.503250+0000","last_clean_scrub_stamp":"2024-01-19T04:55:18.503250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:34:37.409469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-19T04:55:20.825450+0000","last_change":"2024-01-19T04:55:18.531059+0000","last_active":"2024-01-19T04:55:20.825450+0000","last_peered":"2024-01-19T04:55:20.825450+0000","last_clean":"2024-01-19T04:55:20.825450+0000","last_became_active":"2024-01-19T04:55:18.530752+0000","last_became_peered":"2024-01-19T04:55:18.530752+0000","last_unstale":"2024-01-19T04:55:20.825450+0000","last_undegraded":"2024-01-19T04:55:20.825450+0000","last_fullsized":"2024-01-19T04:55:20.825450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:17.497959+0000","last_clean_scrub_stamp":"2024-01-19T04:55:17.497959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:04:18.004474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:55:31.656 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-19T04:55:31.656 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-19T04:55:31.657 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-19T04:55:31.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-19T04:55:31.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.793+0000 7f1329f94640 1 Processor -- start 2024-01-19T04:55:31.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.793+0000 7f1329f94640 1 -- start start 2024-01-19T04:55:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1329f94640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13240d08a0 0x7f132415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1329f94640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f132406ecb0 0x7f13240cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1329f94640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13240d0040 con 0x7f132406e8d0 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1329f94640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132415c800 con 0x7f13240d08a0 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1329f94640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13240cd500 con 0x7f132406ecb0 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1322ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13240d08a0 0x7f132415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1322ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13240d08a0 0x7f132415c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43148/0 (socket says 172.21.15.31:43148) 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.794+0000 7f1322ffd640 1 -- 172.21.15.31:0/360746662 learned_addr learned my addr 172.21.15.31:0/360746662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.795+0000 7f1323fff640 1 --2- 172.21.15.31:0/360746662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f132406ecb0 0x7f13240cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.795+0000 7f1322ffd640 1 -- 172.21.15.31:0/360746662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f132406ecb0 msgr2=0x7f13240cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.795+0000 7f1322ffd640 1 --2- 172.21.15.31:0/360746662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f132406ecb0 0x7f13240cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.795+0000 7f1322ffd640 1 -- 172.21.15.31:0/360746662 >> v1:172.21.15.137:6789/0 conn(0x7f132406e8d0 legacy=0x7f13240cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.795+0000 7f1322ffd640 1 -- 172.21.15.31:0/360746662 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240cb800 con 0x7f13240d08a0 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.795+0000 7f1322ffd640 1 --2- 172.21.15.31:0/360746662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13240d08a0 0x7f132415c2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1318009870 tx=0x7f131802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c2a1e60b1de7bf server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f13227fc640 1 -- 172.21.15.31:0/360746662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318031640 con 0x7f13240d08a0 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f13227fc640 1 -- 172.21.15.31:0/360746662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13180317a0 con 0x7f13240d08a0 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f13227fc640 1 -- 172.21.15.31:0/360746662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318031b30 con 0x7f13240d08a0 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 -- 172.21.15.31:0/360746662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13240d08a0 msgr2=0x7f132415c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 --2- 172.21.15.31:0/360746662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13240d08a0 0x7f132415c2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1318009870 tx=0x7f131802b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 -- 172.21.15.31:0/360746662 shutdown_connections 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 --2- 172.21.15.31:0/360746662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f132406ecb0 0x7f13240cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 --2- 172.21.15.31:0/360746662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13240d08a0 0x7f132415c2c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 -- 172.21.15.31:0/360746662 >> 172.21.15.31:0/360746662 conn(0x7f13240b2c60 msgr2=0x7f13240b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 -- 172.21.15.31:0/360746662 shutdown_connections 2024-01-19T04:55:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.796+0000 7f1329f94640 1 -- 172.21.15.31:0/360746662 wait complete. 2024-01-19T04:55:31.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.797+0000 7f1329f94640 1 Processor -- start 2024-01-19T04:55:31.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.797+0000 7f1329f94640 1 -- start start 2024-01-19T04:55:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1329f94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f132406e8d0 0x7f132406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1329f94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f132406ecb0 0x7f13240c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1329f94640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13240d08a0 0x7f13241551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1329f94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132406d060 con 0x7f132406e8d0 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1329f94640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132406bfc0 con 0x7f132406ecb0 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1329f94640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132406c2a0 con 0x7f13240d08a0 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f13237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f132406e8d0 0x7f132406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f13237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f132406e8d0 0x7f132406b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43164/0 (socket says 172.21.15.31:43164) 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f13237fe640 1 -- 172.21.15.31:0/4232357101 learned_addr learned my addr 172.21.15.31:0/4232357101 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1322ffd640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f132406ecb0 0x7f13240c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.798+0000 7f1323fff640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13240d08a0 0x7f13241551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f13237fe640 1 -- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13240d08a0 msgr2=0x7f13241551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f13237fe640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13240d08a0 0x7f13241551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f13237fe640 1 -- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f132406ecb0 msgr2=0x7f13240c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f13237fe640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f132406ecb0 0x7f13240c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f13237fe640 1 -- 172.21.15.31:0/4232357101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180094d0 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f13237fe640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f132406e8d0 0x7f132406b6c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f131400c400 tx=0x7f131400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f130bfff640 1 -- 172.21.15.31:0/4232357101 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140052d0 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f130bfff640 1 -- 172.21.15.31:0/4232357101 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1314005450 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132406c580 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f130bfff640 1 -- 172.21.15.31:0/4232357101 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314007260 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.799+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132406cb40 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.800+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1324074a80 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.800+0000 7f130bfff640 1 -- 172.21.15.31:0/4232357101 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f13140073c0 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.801+0000 7f130bfff640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f131006aed0 0x7f131006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.801+0000 7f130bfff640 1 -- 172.21.15.31:0/4232357101 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13140889b0 con 0x7f132406e8d0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.801+0000 7f1322ffd640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f131006aed0 0x7f131006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.801+0000 7f1322ffd640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f131006aed0 0x7f131006d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1318036000 tx=0x7f131803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:31.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:31.805+0000 7f130bfff640 1 -- 172.21.15.31:0/4232357101 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1314053910 con 0x7f132406e8d0 2024-01-19T04:55:32.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.022+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f13240696e0 con 0x7f132406e8d0 2024-01-19T04:55:32.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.022+0000 7f130bfff640 1 -- 172.21.15.31:0/4232357101 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1314056f30 con 0x7f132406e8d0 2024-01-19T04:55:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-19T04:55:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f131006aed0 msgr2=0x7f131006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f131006aed0 0x7f131006d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1318036000 tx=0x7f131803c040 comp rx=0 tx=0).stop 2024-01-19T04:55:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f132406e8d0 msgr2=0x7f132406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f132406e8d0 0x7f132406b6c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f131400c400 tx=0x7f131400c8c0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 shutdown_connections 2024-01-19T04:55:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f131006aed0 0x7f131006d390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13240d08a0 0x7f13241551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f132406ecb0 0x7f13240c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 --2- 172.21.15.31:0/4232357101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f132406e8d0 0x7f132406b6c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 >> 172.21.15.31:0/4232357101 conn(0x7f13240b2c60 msgr2=0x7f13240d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 shutdown_connections 2024-01-19T04:55:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.024+0000 7f1329f94640 1 -- 172.21.15.31:0/4232357101 wait complete. 2024-01-19T04:55:32.034 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-19T04:55:32.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:32.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.136+0000 7f114ca9b640 1 Processor -- start 2024-01-19T04:55:32.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.136+0000 7f114ca9b640 1 -- start start 2024-01-19T04:55:32.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f114ca9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11480b3030 0x7f1148160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f114ca9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1148071df0 0x7f11481614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f114ca9b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11480b3510 con 0x7f114815d300 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f114ca9b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11480b37f0 con 0x7f1148071df0 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f114ca9b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11480b3ad0 con 0x7f11480b3030 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f1146d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1148071df0 0x7f11481614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f114659c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f114815d300 0x7f114806ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59248/0 (socket says 172.21.15.31:59248) 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f1146d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1148071df0 0x7f11481614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43180/0 (socket says 172.21.15.31:43180) 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f114659c640 1 -- 172.21.15.31:0/223370787 learned_addr learned my addr 172.21.15.31:0/223370787 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f113ffff640 1 --2- 172.21.15.31:0/223370787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11480b3030 0x7f1148160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f1146d9d640 1 -- 172.21.15.31:0/223370787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11480b3030 msgr2=0x7f1148160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f1146d9d640 1 --2- 172.21.15.31:0/223370787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11480b3030 0x7f1148160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f1146d9d640 1 -- 172.21.15.31:0/223370787 >> v1:172.21.15.31:6789/0 conn(0x7f114815d300 legacy=0x7f114806ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f1145d9b640 1 -- 172.21.15.31:0/223370787 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051369030 0 0) 0x7f11480b3510 con 0x7f114815d300 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.137+0000 7f1146d9d640 1 -- 172.21.15.31:0/223370787 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114806a650 con 0x7f1148071df0 2024-01-19T04:55:32.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.138+0000 7f1146d9d640 1 --2- 172.21.15.31:0/223370787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f11481614a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1140009d90 tx=0x7f114002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7dd8308e01ac58 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.138+0000 7f1145d9b640 1 -- 172.21.15.31:0/223370787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114002cc50 con 0x7f1148071df0 2024-01-19T04:55:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.138+0000 7f1145d9b640 1 -- 172.21.15.31:0/223370787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114002cdb0 con 0x7f1148071df0 2024-01-19T04:55:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.138+0000 7f1145d9b640 1 -- 172.21.15.31:0/223370787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140034660 con 0x7f1148071df0 2024-01-19T04:55:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.138+0000 7f114ca9b640 1 -- 172.21.15.31:0/223370787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 msgr2=0x7f11481614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.138+0000 7f114ca9b640 1 --2- 172.21.15.31:0/223370787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f11481614a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1140009d90 tx=0x7f114002bea0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.138+0000 7f114ca9b640 1 -- 172.21.15.31:0/223370787 shutdown_connections 2024-01-19T04:55:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.139+0000 7f114ca9b640 1 --2- 172.21.15.31:0/223370787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f11481614a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.139+0000 7f114ca9b640 1 --2- 172.21.15.31:0/223370787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11480b3030 0x7f1148160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.139+0000 7f114ca9b640 1 -- 172.21.15.31:0/223370787 >> 172.21.15.31:0/223370787 conn(0x7f114806d750 msgr2=0x7f114806db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.139+0000 7f114ca9b640 1 -- 172.21.15.31:0/223370787 shutdown_connections 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.139+0000 7f114ca9b640 1 -- 172.21.15.31:0/223370787 wait complete. 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.140+0000 7f114ca9b640 1 Processor -- start 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.140+0000 7f114ca9b640 1 -- start start 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.140+0000 7f114ca9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f114815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.140+0000 7f114ca9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11480b3030 0x7f11480b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114ca9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f114815d300 0x7f1148152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114ca9b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1148142960 con 0x7f1148071df0 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114ca9b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114813f770 con 0x7f11480b3030 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114ca9b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1148141a90 con 0x7f114815d300 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f114815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f114815c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43188/0 (socket says 172.21.15.31:43188) 2024-01-19T04:55:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 -- 172.21.15.31:0/1555825754 learned_addr learned my addr 172.21.15.31:0/1555825754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f1146d9d640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f114815d300 0x7f1148152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f113ffff640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11480b3030 0x7f11480b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 -- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f114815d300 msgr2=0x7f1148152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f114815d300 0x7f1148152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 -- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11480b3030 msgr2=0x7f11480b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11480b3030 0x7f11480b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 -- 172.21.15.31:0/1555825754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140009a70 con 0x7f1148071df0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f1146d9d640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f114815d300 0x7f1148152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114659c640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f114815c140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f113800c460 tx=0x7f113800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f113effd640 1 -- 172.21.15.31:0/1555825754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380074e0 con 0x7f1148071df0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f113effd640 1 -- 172.21.15.31:0/1555825754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1138007660 con 0x7f1148071df0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f113effd640 1 -- 172.21.15.31:0/1555825754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138004a20 con 0x7f1148071df0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1148141d40 con 0x7f1148071df0 2024-01-19T04:55:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.141+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1148142580 con 0x7f1148071df0 2024-01-19T04:55:32.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.142+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11480696e0 con 0x7f1148071df0 2024-01-19T04:55:32.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.143+0000 7f113effd640 1 -- 172.21.15.31:0/1555825754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1138004b80 con 0x7f1148071df0 2024-01-19T04:55:32.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.143+0000 7f113effd640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f112806aed0 0x7f112806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.143+0000 7f113effd640 1 -- 172.21.15.31:0/1555825754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11380887b0 con 0x7f1148071df0 2024-01-19T04:55:32.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.143+0000 7f113ffff640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f112806aed0 0x7f112806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.144+0000 7f113ffff640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f112806aed0 0x7f112806d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f114815b970 tx=0x7f1130008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.147+0000 7f113effd640 1 -- 172.21.15.31:0/1555825754 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1138053710 con 0x7f1148071df0 2024-01-19T04:55:32.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.325+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f114813fa00 con 0x7f1148071df0 2024-01-19T04:55:32.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.327+0000 7f113effd640 1 -- 172.21.15.31:0/1555825754 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f1138018020 con 0x7f1148071df0 2024-01-19T04:55:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:32.326 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:32.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f112806aed0 msgr2=0x7f112806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f112806aed0 0x7f112806d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f114815b970 tx=0x7f1130008040 comp rx=0 tx=0).stop 2024-01-19T04:55:32.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 msgr2=0x7f114815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f114815c140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f113800c460 tx=0x7f113800c920 comp rx=0 tx=0).stop 2024-01-19T04:55:32.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 shutdown_connections 2024-01-19T04:55:32.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f112806aed0 0x7f112806d390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f114815d300 0x7f1148152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11480b3030 0x7f11480b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 --2- 172.21.15.31:0/1555825754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1148071df0 0x7f114815c140 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 >> 172.21.15.31:0/1555825754 conn(0x7f114806d750 msgr2=0x7f114815a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:32.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 shutdown_connections 2024-01-19T04:55:32.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.328+0000 7f114ca9b640 1 -- 172.21.15.31:0/1555825754 wait complete. 2024-01-19T04:55:32.339 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:32.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.443+0000 7fcb90305640 1 Processor -- start 2024-01-19T04:55:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.443+0000 7fcb90305640 1 -- start start 2024-01-19T04:55:32.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb90305640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb88161860 0x7fcb88168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb90305640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb880cf9f0 con 0x7fcb880cf570 2024-01-19T04:55:32.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb90305640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb880cfcd0 con 0x7fcb8815d3a0 2024-01-19T04:55:32.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb90305640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880cffb0 con 0x7fcb88161860 2024-01-19T04:55:32.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb8e8a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb88161860 0x7fcb88168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb8e8a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb88161860 0x7fcb88168490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43200/0 (socket says 172.21.15.31:43200) 2024-01-19T04:55:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/886216789 learned_addr learned my addr 172.21.15.31:0/886216789 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb8d8a0640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcb880cf570 0x7fcb88161150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59250/0 (socket says 172.21.15.31:59250) 2024-01-19T04:55:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/886216789 >> v1:172.21.15.137:6789/0 conn(0x7fcb8815d3a0 legacy=0x7fcb8806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/886216789 >> v1:172.21.15.31:6789/0 conn(0x7fcb880cf570 legacy=0x7fcb88161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.444+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/886216789 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb880d0290 con 0x7fcb88161860 2024-01-19T04:55:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.445+0000 7fcb8e8a2640 1 --2- 172.21.15.31:0/886216789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb88168490 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009e50 tx=0x7fcb7802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cc9b8600bba3c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.445+0000 7fcb8d09f640 1 -- 172.21.15.31:0/886216789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7802ca80 con 0x7fcb88161860 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.445+0000 7fcb8d09f640 1 -- 172.21.15.31:0/886216789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7802cbe0 con 0x7fcb88161860 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.445+0000 7fcb8d09f640 1 -- 172.21.15.31:0/886216789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb780343f0 con 0x7fcb88161860 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.446+0000 7fcb90305640 1 -- 172.21.15.31:0/886216789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 msgr2=0x7fcb88168490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.446+0000 7fcb90305640 1 --2- 172.21.15.31:0/886216789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb88168490 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009e50 tx=0x7fcb7802bee0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.446+0000 7fcb90305640 1 -- 172.21.15.31:0/886216789 shutdown_connections 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.446+0000 7fcb90305640 1 --2- 172.21.15.31:0/886216789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb88168490 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.446+0000 7fcb90305640 1 -- 172.21.15.31:0/886216789 >> 172.21.15.31:0/886216789 conn(0x7fcb880d1100 msgr2=0x7fcb880d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.446+0000 7fcb90305640 1 -- 172.21.15.31:0/886216789 shutdown_connections 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.446+0000 7fcb90305640 1 -- 172.21.15.31:0/886216789 wait complete. 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.447+0000 7fcb90305640 1 Processor -- start 2024-01-19T04:55:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.447+0000 7fcb90305640 1 -- start start 2024-01-19T04:55:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb90305640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb880cf570 0x7fcb880b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb90305640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb8815d3a0 0x7fcb8815d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb90305640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb8815dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb90305640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b1ae0 con 0x7fcb88161860 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb90305640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b18e0 con 0x7fcb880cf570 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb90305640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8815e2f0 con 0x7fcb8815d3a0 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb8815dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb8815dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43210/0 (socket says 172.21.15.31:43210) 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/4028211465 learned_addr learned my addr 172.21.15.31:0/4028211465 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8d8a0640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb8815d3a0 0x7fcb8815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e0a1640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb880cf570 0x7fcb880b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb8815d3a0 msgr2=0x7fcb8815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb8815d3a0 0x7fcb8815d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb880cf570 msgr2=0x7fcb880b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb880cf570 0x7fcb880b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 -- 172.21.15.31:0/4028211465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78009b30 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8d8a0640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb8815d3a0 0x7fcb8815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb8e8a2640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb8815dcc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb78036040 tx=0x7fcb78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb86ffd640 1 -- 172.21.15.31:0/4028211465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7803e450 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb86ffd640 1 -- 172.21.15.31:0/4028211465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7803e5d0 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.448+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8815e5d0 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.449+0000 7fcb86ffd640 1 -- 172.21.15.31:0/4028211465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7803e980 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.449+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8815eb90 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.449+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb880c7980 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.450+0000 7fcb86ffd640 1 -- 172.21.15.31:0/4028211465 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fcb7802cd50 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.450+0000 7fcb86ffd640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcb7406af00 0x7fcb7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.450+0000 7fcb86ffd640 1 -- 172.21.15.31:0/4028211465 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb780abeb0 con 0x7fcb88161860 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.451+0000 7fcb8e0a1640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcb7406af00 0x7fcb7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.451+0000 7fcb8e0a1640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcb7406af00 0x7fcb7406d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcb80004e50 tx=0x7fcb80009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.455+0000 7fcb86ffd640 1 -- 172.21.15.31:0/4028211465 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fcb78076e10 con 0x7fcb88161860 2024-01-19T04:55:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.632+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb880b1c20 con 0x7fcb88161860 2024-01-19T04:55:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.632+0000 7fcb86ffd640 1 -- 172.21.15.31:0/4028211465 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb7807a430 con 0x7fcb88161860 2024-01-19T04:55:32.632 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcb7406af00 msgr2=0x7fcb7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcb7406af00 0x7fcb7406d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcb80004e50 tx=0x7fcb80009040 comp rx=0 tx=0).stop 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 msgr2=0x7fcb8815dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb8815dcc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb78036040 tx=0x7fcb78004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 shutdown_connections 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcb7406af00 0x7fcb7406d3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88161860 0x7fcb8815dcc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb8815d3a0 0x7fcb8815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 --2- 172.21.15.31:0/4028211465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb880cf570 0x7fcb880b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 >> 172.21.15.31:0/4028211465 conn(0x7fcb880d1100 msgr2=0x7fcb880d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 shutdown_connections 2024-01-19T04:55:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.634+0000 7fcb90305640 1 -- 172.21.15.31:0/4028211465 wait complete. 2024-01-19T04:55:32.644 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.748+0000 7fa54dd1d640 1 Processor -- start 2024-01-19T04:55:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.748+0000 7fa54dd1d640 1 -- start start 2024-01-19T04:55:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa54dd1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 0x7fa5480cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa54dd1d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54806cf80 con 0x7fa54815dc00 2024-01-19T04:55:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa54dd1d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5480cb680 con 0x7fa54806c730 2024-01-19T04:55:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa54dd1d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480cbf20 con 0x7fa54809a160 2024-01-19T04:55:32.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa5477fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa54815dc00 0x7fa548099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59252/0 (socket says 172.21.15.31:59252) 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa5477fe640 1 -- 172.21.15.31:0/1758612343 learned_addr learned my addr 172.21.15.31:0/1758612343 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa546ffd640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 0x7fa5480cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394360237 0 0) 0x7fa54806cf80 con 0x7fa54815dc00 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.749+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa530003520 con 0x7fa54815dc00 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 66626763 0 0) 0x7fa530003520 con 0x7fa54815dc00 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 msgr2=0x7fa5480cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 0x7fa5480cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 >> v1:172.21.15.137:6789/0 conn(0x7fa54806c730 legacy=0x7fa5480cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480adca0 con 0x7fa54815dc00 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fa538001670 con 0x7fa54815dc00 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 >> v1:172.21.15.31:6789/0 conn(0x7fa54815dc00 legacy=0x7fa548099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5300039a0 0x7fa530003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 0x7fa5300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa530004d00 0x7fa530005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480cbf20 con 0x7fa530004d00 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53000ba30 con 0x7fa5300039a0 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53000bbf0 con 0x7fa54809a160 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa547fff640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa530004d00 0x7fa530005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa5477fe640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5300039a0 0x7fa530003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa546ffd640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 0x7fa5300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa547fff640 1 -- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 msgr2=0x7fa5300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa547fff640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 0x7fa5300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa547fff640 1 -- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5300039a0 msgr2=0x7fa530003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa547fff640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5300039a0 0x7fa530003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa547fff640 1 -- 172.21.15.31:0/1758612343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530003520 con 0x7fa530004d00 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.750+0000 7fa546ffd640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa54809a160 0x7fa5300047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.751+0000 7fa547fff640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa530004d00 0x7fa530005180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa540001e20 tx=0x7fa5300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d5b76854a1d9dc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.751+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540003b70 con 0x7fa530004d00 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.751+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa540003cd0 con 0x7fa530004d00 2024-01-19T04:55:32.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.751+0000 7fa5467fc640 1 -- 172.21.15.31:0/1758612343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540005310 con 0x7fa530004d00 2024-01-19T04:55:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.751+0000 7fa54dd1d640 1 -- 172.21.15.31:0/1758612343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa530004d00 msgr2=0x7fa530005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.751+0000 7fa54dd1d640 1 --2- 172.21.15.31:0/1758612343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa530004d00 0x7fa530005180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa540001e20 tx=0x7fa5300057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.751+0000 7fa547fff640 1 -- 172.21.15.31:0/1758612343 reap_dead start 2024-01-19T04:55:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.752+0000 7fa54dd1d640 1 -- 172.21.15.31:0/1758612343 shutdown_connections 2024-01-19T04:55:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.752+0000 7fa54dd1d640 1 -- 172.21.15.31:0/1758612343 >> 172.21.15.31:0/1758612343 conn(0x7fa5480b2b20 msgr2=0x7fa5480b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.752+0000 7fa54dd1d640 1 -- 172.21.15.31:0/1758612343 shutdown_connections 2024-01-19T04:55:32.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.752+0000 7fa54dd1d640 1 -- 172.21.15.31:0/1758612343 wait complete. 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.753+0000 7fa54dd1d640 1 Processor -- start 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.753+0000 7fa54dd1d640 1 -- start start 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5481595c0 0x7fa54815e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa54815eea0 0x7fa5480b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5480b5220 0x7fa54815b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815f320 con 0x7fa54815eea0 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b6ab0 con 0x7fa5481595c0 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b6620 con 0x7fa5480b5220 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa54815eea0 0x7fa5480b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa54815eea0 0x7fa5480b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43234/0 (socket says 172.21.15.31:43234) 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 -- 172.21.15.31:0/430164268 learned_addr learned my addr 172.21.15.31:0/430164268 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:32.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa5477fe640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5481595c0 0x7fa54815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa547fff640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5480b5220 0x7fa54815b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 -- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5480b5220 msgr2=0x7fa54815b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5480b5220 0x7fa54815b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 -- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5481595c0 msgr2=0x7fa54815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5481595c0 0x7fa54815e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 -- 172.21.15.31:0/430164268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480b5a10 con 0x7fa54815eea0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa5477fe640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5481595c0 0x7fa54815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa547fff640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5480b5220 0x7fa54815b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa546ffd640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa54815eea0 0x7fa5480b4ce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa53c011730 tx=0x7fa53c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa52ffff640 1 -- 172.21.15.31:0/430164268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c00f9c0 con 0x7fa54815eea0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa52ffff640 1 -- 172.21.15.31:0/430164268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c00fb20 con 0x7fa54815eea0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5480b5cf0 con 0x7fa54815eea0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa52ffff640 1 -- 172.21.15.31:0/430164268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c00fed0 con 0x7fa54815eea0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.754+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5480b62b0 con 0x7fa54815eea0 2024-01-19T04:55:32.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.755+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa548074900 con 0x7fa54815eea0 2024-01-19T04:55:32.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.756+0000 7fa52ffff640 1 -- 172.21.15.31:0/430164268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa53c015460 con 0x7fa54815eea0 2024-01-19T04:55:32.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.756+0000 7fa52ffff640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa53406aed0 0x7fa53406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:32.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.756+0000 7fa52ffff640 1 -- 172.21.15.31:0/430164268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa53c089f20 con 0x7fa54815eea0 2024-01-19T04:55:32.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.756+0000 7fa5477fe640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa53406aed0 0x7fa53406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:32.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.757+0000 7fa5477fe640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa53406aed0 0x7fa53406d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa538007c60 tx=0x7fa538009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:32.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.760+0000 7fa52ffff640 1 -- 172.21.15.31:0/430164268 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa53c054df0 con 0x7fa54815eea0 2024-01-19T04:55:32.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.938+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa54806fc10 con 0x7fa54815eea0 2024-01-19T04:55:32.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.938+0000 7fa52ffff640 1 -- 172.21.15.31:0/430164268 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa53c058410 con 0x7fa54815eea0 2024-01-19T04:55:32.938 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:32.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa53406aed0 msgr2=0x7fa53406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa53406aed0 0x7fa53406d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa538007c60 tx=0x7fa538009040 comp rx=0 tx=0).stop 2024-01-19T04:55:32.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa54815eea0 msgr2=0x7fa5480b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:32.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa54815eea0 0x7fa5480b4ce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa53c011730 tx=0x7fa53c011bf0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 shutdown_connections 2024-01-19T04:55:32.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa53406aed0 0x7fa53406d390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5480b5220 0x7fa54815b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa54815eea0 0x7fa5480b4ce0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 --2- 172.21.15.31:0/430164268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5481595c0 0x7fa54815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:32.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.940+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 >> 172.21.15.31:0/430164268 conn(0x7fa5480b2b20 msgr2=0x7fa5480cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:32.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.941+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 shutdown_connections 2024-01-19T04:55:32.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:32.941+0000 7fa54dd1d640 1 -- 172.21.15.31:0/430164268 wait complete. 2024-01-19T04:55:32.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.058+0000 7fb20591d640 1 Processor -- start 2024-01-19T04:55:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.058+0000 7fb20591d640 1 -- start start 2024-01-19T04:55:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb20591d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb2000ce090 0x7fb2000d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb20591d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000cd2e0 0x7fb2000b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb20591d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2000cce50 con 0x7fb20015b6b0 2024-01-19T04:55:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb20591d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2000d03b0 con 0x7fb2000ce090 2024-01-19T04:55:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb20591d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000cc240 con 0x7fb2000cd2e0 2024-01-19T04:55:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000cd2e0 0x7fb2000b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb1fe7fc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb2000ce090 0x7fb2000d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000cd2e0 0x7fb2000b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36542/0 (socket says 172.21.15.31:36542) 2024-01-19T04:55:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb1fe7fc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb2000ce090 0x7fb2000d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56342/0 (socket says 172.21.15.31:56342) 2024-01-19T04:55:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.059+0000 7fb1ff7fe640 1 -- 172.21.15.31:0/1362368201 learned_addr learned my addr 172.21.15.31:0/1362368201 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.060+0000 7fb1fe7fc640 1 -- 172.21.15.31:0/1362368201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000cd2e0 msgr2=0x7fb2000b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.060+0000 7fb1fe7fc640 1 --2- 172.21.15.31:0/1362368201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000cd2e0 0x7fb2000b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.060+0000 7fb1fe7fc640 1 -- 172.21.15.31:0/1362368201 >> v1:172.21.15.137:6789/0 conn(0x7fb20015b6b0 legacy=0x7fb2000cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.060+0000 7fb1fe7fc640 1 -- 172.21.15.31:0/1362368201 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2000cc520 con 0x7fb2000ce090 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.060+0000 7fb1fe7fc640 1 --2- 172.21.15.31:0/1362368201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2000ce090 0x7fb2000d1c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80097a0 tx=0x7fb1e802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd56fd2e5eb75e0f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.060+0000 7fb1fdffb640 1 -- 172.21.15.31:0/1362368201 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8030610 con 0x7fb2000ce090 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb1fdffb640 1 -- 172.21.15.31:0/1362368201 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e8030770 con 0x7fb2000ce090 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb1fdffb640 1 -- 172.21.15.31:0/1362368201 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8030b00 con 0x7fb2000ce090 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 -- 172.21.15.31:0/1362368201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2000ce090 msgr2=0x7fb2000d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 --2- 172.21.15.31:0/1362368201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2000ce090 0x7fb2000d1c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80097a0 tx=0x7fb1e802b960 comp rx=0 tx=0).stop 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 -- 172.21.15.31:0/1362368201 shutdown_connections 2024-01-19T04:55:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 --2- 172.21.15.31:0/1362368201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000cd2e0 0x7fb2000b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 --2- 172.21.15.31:0/1362368201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2000ce090 0x7fb2000d1c20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 -- 172.21.15.31:0/1362368201 >> 172.21.15.31:0/1362368201 conn(0x7fb2000d1100 msgr2=0x7fb2000d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 -- 172.21.15.31:0/1362368201 shutdown_connections 2024-01-19T04:55:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.061+0000 7fb20591d640 1 -- 172.21.15.31:0/1362368201 wait complete. 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.062+0000 7fb20591d640 1 Processor -- start 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.062+0000 7fb20591d640 1 -- start start 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb20591d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2000cd2e0 0x7fb2000c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb20591d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000ce090 0x7fb200154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb20591d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb20015b6b0 0x7fb2000c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb20591d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2001546c0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb20591d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000c5280 con 0x7fb20015b6b0 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb20591d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000c74f0 con 0x7fb2000ce090 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb1feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2000cd2e0 0x7fb2000c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb1feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2000cd2e0 0x7fb2000c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43246/0 (socket says 172.21.15.31:43246) 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb1feffd640 1 -- 172.21.15.31:0/1983505694 learned_addr learned my addr 172.21.15.31:0/1983505694 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb1ff7fe640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb20015b6b0 0x7fb2000c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.063+0000 7fb1fe7fc640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000ce090 0x7fb200154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1feffd640 1 -- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000ce090 msgr2=0x7fb200154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1feffd640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000ce090 0x7fb200154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1feffd640 1 -- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb20015b6b0 msgr2=0x7fb2000c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1feffd640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb20015b6b0 0x7fb2000c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1feffd640 1 -- 172.21.15.31:0/1983505694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e8009400 con 0x7fb2000cd2e0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1fe7fc640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000ce090 0x7fb200154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1feffd640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2000cd2e0 0x7fb2000c4c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c730 tx=0x7fb1f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1f7fff640 1 -- 172.21.15.31:0/1983505694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0006da0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1f7fff640 1 -- 172.21.15.31:0/1983505694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f000ce40 con 0x7fb2000cd2e0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2000c77d0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb1f7fff640 1 -- 172.21.15.31:0/1983505694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0003c00 con 0x7fb2000cd2e0 2024-01-19T04:55:33.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.064+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2000c5ca0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.065+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2000c7cf0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.065+0000 7fb1f7fff640 1 -- 172.21.15.31:0/1983505694 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb1f000a7f0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.066+0000 7fb1f7fff640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1e006af00 0x7fb1e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.066+0000 7fb1f7fff640 1 -- 172.21.15.31:0/1983505694 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1f00890d0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.066+0000 7fb1fe7fc640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1e006af00 0x7fb1e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.066+0000 7fb1fe7fc640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1e006af00 0x7fb1e006d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8036000 tx=0x7fb1e8008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.070+0000 7fb1f7fff640 1 -- 172.21.15.31:0/1983505694 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb2000c7cf0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.246+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb200074900 con 0x7fb2000cd2e0 2024-01-19T04:55:33.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.247+0000 7fb1f7fff640 1 -- 172.21.15.31:0/1983505694 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1f0057ac0 con 0x7fb2000cd2e0 2024-01-19T04:55:33.246 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1e006af00 msgr2=0x7fb1e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1e006af00 0x7fb1e006d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8036000 tx=0x7fb1e8008030 comp rx=0 tx=0).stop 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2000cd2e0 msgr2=0x7fb2000c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2000cd2e0 0x7fb2000c4c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c730 tx=0x7fb1f000cbf0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 shutdown_connections 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb1e006af00 0x7fb1e006d3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb20015b6b0 0x7fb2000c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb2000ce090 0x7fb200154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 --2- 172.21.15.31:0/1983505694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2000cd2e0 0x7fb2000c4c30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.248+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 >> 172.21.15.31:0/1983505694 conn(0x7fb2000d1100 msgr2=0x7fb20015d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.249+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 shutdown_connections 2024-01-19T04:55:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.249+0000 7fb20591d640 1 -- 172.21.15.31:0/1983505694 wait complete. 2024-01-19T04:55:33.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:55:33.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.360+0000 7fa23adde640 1 Processor -- start 2024-01-19T04:55:33.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.360+0000 7fa23adde640 1 -- start start 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23adde640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa23415f0c0 0x7fa23415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23adde640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23406bf30 con 0x7fa23415eb70 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23adde640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340cb9e0 con 0x7fa23415d8a0 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23adde640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23415fa30 con 0x7fa23415f0c0 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23937b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa23415f0c0 0x7fa23415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23937b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa23415f0c0 0x7fa23415f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43256/0 (socket says 172.21.15.31:43256) 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23937b640 1 -- 172.21.15.31:0/151342282 learned_addr learned my addr 172.21.15.31:0/151342282 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa233fff640 1 --1- 172.21.15.31:0/151342282 >> v1:172.21.15.31:6789/0 conn(0x7fa23415eb70 0x7fa2340cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59260/0 (socket says 172.21.15.31:59260) 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23937b640 1 -- 172.21.15.31:0/151342282 >> v1:172.21.15.137:6789/0 conn(0x7fa23415d8a0 legacy=0x7fa23406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23937b640 1 -- 172.21.15.31:0/151342282 >> v1:172.21.15.31:6789/0 conn(0x7fa23415eb70 legacy=0x7fa2340cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa23937b640 1 -- 172.21.15.31:0/151342282 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340cbcc0 con 0x7fa23415f0c0 2024-01-19T04:55:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.361+0000 7fa2337fe640 1 -- 172.21.15.31:0/151342282 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850425822 0 0) 0x7fa23406bf30 con 0x7fa23415eb70 2024-01-19T04:55:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.362+0000 7fa23937b640 1 --2- 172.21.15.31:0/151342282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415f0c0 0x7fa23415f4f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009e50 tx=0x7fa22c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f6bf0167125abc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.362+0000 7fa2337fe640 1 -- 172.21.15.31:0/151342282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c02ca80 con 0x7fa23415f0c0 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.362+0000 7fa2337fe640 1 -- 172.21.15.31:0/151342282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c02cbe0 con 0x7fa23415f0c0 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.362+0000 7fa2337fe640 1 -- 172.21.15.31:0/151342282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0343f0 con 0x7fa23415f0c0 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.362+0000 7fa23adde640 1 -- 172.21.15.31:0/151342282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415f0c0 msgr2=0x7fa23415f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.362+0000 7fa23adde640 1 --2- 172.21.15.31:0/151342282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415f0c0 0x7fa23415f4f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009e50 tx=0x7fa22c02bee0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.363+0000 7fa23adde640 1 -- 172.21.15.31:0/151342282 shutdown_connections 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.363+0000 7fa23adde640 1 --2- 172.21.15.31:0/151342282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415f0c0 0x7fa23415f4f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.363+0000 7fa23adde640 1 -- 172.21.15.31:0/151342282 >> 172.21.15.31:0/151342282 conn(0x7fa23406abe0 msgr2=0x7fa234066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.363+0000 7fa23adde640 1 -- 172.21.15.31:0/151342282 shutdown_connections 2024-01-19T04:55:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.363+0000 7fa23adde640 1 -- 172.21.15.31:0/151342282 wait complete. 2024-01-19T04:55:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.364+0000 7fa23adde640 1 Processor -- start 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.364+0000 7fa23adde640 1 -- start start 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415d8a0 0x7fa2341f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa23415eb70 0x7fa2340b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa23415f0c0 0x7fa2340d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341f4f70 con 0x7fa23415d8a0 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341f10f0 con 0x7fa23415f0c0 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340b6bf0 con 0x7fa23415eb70 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415d8a0 0x7fa2341f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415d8a0 0x7fa2341f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43272/0 (socket says 172.21.15.31:43272) 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 -- 172.21.15.31:0/2140474811 learned_addr learned my addr 172.21.15.31:0/2140474811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa233fff640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa23415eb70 0x7fa2340b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23937b640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa23415f0c0 0x7fa2340d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 -- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa23415eb70 msgr2=0x7fa2340b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa23415eb70 0x7fa2340b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 -- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa23415f0c0 msgr2=0x7fa2340d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa23415f0c0 0x7fa2340d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 -- 172.21.15.31:0/2140474811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c009b30 con 0x7fa23415d8a0 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa238b7a640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415d8a0 0x7fa2341f0bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa22800b410 tx=0x7fa22800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa2317fa640 1 -- 172.21.15.31:0/2140474811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228011830 con 0x7fa23415d8a0 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa2317fa640 1 -- 172.21.15.31:0/2140474811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228011990 con 0x7fa23415d8a0 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2341f48b0 con 0x7fa23415d8a0 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa2317fa640 1 -- 172.21.15.31:0/2140474811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228011d40 con 0x7fa23415d8a0 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.365+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2340b6ed0 con 0x7fa23415d8a0 2024-01-19T04:55:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.366+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2340ccb50 con 0x7fa23415d8a0 2024-01-19T04:55:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.367+0000 7fa2317fa640 1 -- 172.21.15.31:0/2140474811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa22801a400 con 0x7fa23415d8a0 2024-01-19T04:55:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.367+0000 7fa2317fa640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa21806aed0 0x7fa21806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.367+0000 7fa2317fa640 1 -- 172.21.15.31:0/2140474811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2280895e0 con 0x7fa23415d8a0 2024-01-19T04:55:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.367+0000 7fa233fff640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa21806aed0 0x7fa21806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.368+0000 7fa233fff640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa21806aed0 0x7fa21806d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2341f6ba0 tx=0x7fa220007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.371+0000 7fa2317fa640 1 -- 172.21.15.31:0/2140474811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa2280545f0 con 0x7fa23415d8a0 2024-01-19T04:55:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.590+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2340696e0 con 0x7fa23415d8a0 2024-01-19T04:55:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.590+0000 7fa2317fa640 1 -- 172.21.15.31:0/2140474811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fa228057c10 con 0x7fa23415d8a0 2024-01-19T04:55:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:33.590 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:55:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:55:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa21806aed0 msgr2=0x7fa21806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa21806aed0 0x7fa21806d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2341f6ba0 tx=0x7fa220007d50 comp rx=0 tx=0).stop 2024-01-19T04:55:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415d8a0 msgr2=0x7fa2341f0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415d8a0 0x7fa2341f0bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa22800b410 tx=0x7fa22800b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 shutdown_connections 2024-01-19T04:55:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa21806aed0 0x7fa21806d390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa23415f0c0 0x7fa2340d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa23415eb70 0x7fa2340b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.592+0000 7fa23adde640 1 --2- 172.21.15.31:0/2140474811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23415d8a0 0x7fa2341f0bb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.593+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 >> 172.21.15.31:0/2140474811 conn(0x7fa23406abe0 msgr2=0x7fa2340b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.593+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 shutdown_connections 2024-01-19T04:55:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.593+0000 7fa23adde640 1 -- 172.21.15.31:0/2140474811 wait complete. 2024-01-19T04:55:33.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.704+0000 7fdd1bf2c640 1 Processor -- start 2024-01-19T04:55:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1bf2c640 1 -- start start 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1bf2c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fdd140d02f0 0x7fdd140b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1bf2c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd140cf150 0x7fdd140b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1bf2c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd140d22b0 con 0x7fdd14161ee0 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1bf2c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd140d41c0 con 0x7fdd140cf150 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1bf2c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd140d1980 con 0x7fdd140d02f0 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1a4c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd140cf150 0x7fdd140b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd19cc8640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd14161ee0 0x7fdd140cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59270/0 (socket says 172.21.15.31:59270) 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd19cc8640 1 -- 172.21.15.31:0/2759926549 learned_addr learned my addr 172.21.15.31:0/2759926549 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.705+0000 7fdd1a4c9640 1 --2- 172.21.15.31:0/2759926549 >> v2:172.21.15.31:3300/0 conn(0x7fdd140cf150 0x7fdd140b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43288/0 (socket says 172.21.15.31:43288) 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.706+0000 7fdd194c7640 1 --2- 172.21.15.31:0/2759926549 >> v2:172.21.15.137:3300/0 conn(0x7fdd140d02f0 0x7fdd140b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.706+0000 7fdd1a4c9640 1 -- 172.21.15.31:0/2759926549 >> v2:172.21.15.137:3300/0 conn(0x7fdd140d02f0 msgr2=0x7fdd140b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.706+0000 7fdd1a4c9640 1 --2- 172.21.15.31:0/2759926549 >> v2:172.21.15.137:3300/0 conn(0x7fdd140d02f0 0x7fdd140b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.706+0000 7fdd1a4c9640 1 -- 172.21.15.31:0/2759926549 >> v1:172.21.15.31:6789/0 conn(0x7fdd14161ee0 legacy=0x7fdd140cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.706+0000 7fdd1a4c9640 1 -- 172.21.15.31:0/2759926549 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd140d1c60 con 0x7fdd140cf150 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.706+0000 7fdd18cc6640 1 -- 172.21.15.31:0/2759926549 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484860683 0 0) 0x7fdd140d22b0 con 0x7fdd14161ee0 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.706+0000 7fdd1a4c9640 1 --2- 172.21.15.31:0/2759926549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140cf150 0x7fdd140b63a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd1406b230 tx=0x7fdd0c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541daa1bfea0dcf9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd18cc6640 1 -- 172.21.15.31:0/2759926549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c02cc80 con 0x7fdd140cf150 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd18cc6640 1 -- 172.21.15.31:0/2759926549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c02cde0 con 0x7fdd140cf150 2024-01-19T04:55:33.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd18cc6640 1 -- 172.21.15.31:0/2759926549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c034660 con 0x7fdd140cf150 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2759926549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140cf150 msgr2=0x7fdd140b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2759926549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140cf150 0x7fdd140b63a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd1406b230 tx=0x7fdd0c02bed0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2759926549 shutdown_connections 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2759926549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140cf150 0x7fdd140b63a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2759926549 >> v2:172.21.15.137:3300/0 conn(0x7fdd140d02f0 0x7fdd140b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2759926549 >> 172.21.15.31:0/2759926549 conn(0x7fdd1409c680 msgr2=0x7fdd140d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2759926549 shutdown_connections 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.707+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2759926549 wait complete. 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.708+0000 7fdd1bf2c640 1 Processor -- start 2024-01-19T04:55:33.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.708+0000 7fdd1bf2c640 1 -- start start 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd1bf2c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd140cf150 0x7fdd141537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd1bf2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140d02f0 0x7fdd141553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd1bf2c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd14161ee0 0x7fdd141558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd1bf2c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140c7960 con 0x7fdd140d02f0 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd1bf2c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140c7c40 con 0x7fdd140cf150 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd1bf2c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140c7f20 con 0x7fdd14161ee0 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140d02f0 0x7fdd141553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140d02f0 0x7fdd141553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43304/0 (socket says 172.21.15.31:43304) 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 -- 172.21.15.31:0/2677758745 learned_addr learned my addr 172.21.15.31:0/2677758745 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd19cc8640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd140cf150 0x7fdd141537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd1a4c9640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd14161ee0 0x7fdd141558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 -- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd14161ee0 msgr2=0x7fdd141558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd14161ee0 0x7fdd141558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 -- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd140cf150 msgr2=0x7fdd141537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd140cf150 0x7fdd141537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.709+0000 7fdd194c7640 1 -- 172.21.15.31:0/2677758745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c009a70 con 0x7fdd140d02f0 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.710+0000 7fdd194c7640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140d02f0 0x7fdd141553a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d800 tx=0x7fdd0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.710+0000 7fdd0a7fc640 1 -- 172.21.15.31:0/2677758745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400b6c0 con 0x7fdd140d02f0 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.710+0000 7fdd0a7fc640 1 -- 172.21.15.31:0/2677758745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0400b820 con 0x7fdd140d02f0 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.710+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd140c8200 con 0x7fdd140d02f0 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.710+0000 7fdd0a7fc640 1 -- 172.21.15.31:0/2677758745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400bbb0 con 0x7fdd140d02f0 2024-01-19T04:55:33.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.710+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd14153ce0 con 0x7fdd140d02f0 2024-01-19T04:55:33.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.711+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd14074a80 con 0x7fdd140d02f0 2024-01-19T04:55:33.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.711+0000 7fdd0a7fc640 1 -- 172.21.15.31:0/2677758745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdd0400bd10 con 0x7fdd140d02f0 2024-01-19T04:55:33.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.712+0000 7fdd0a7fc640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdcf806aed0 0x7fdcf806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:33.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.712+0000 7fdd0a7fc640 1 -- 172.21.15.31:0/2677758745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd04089be0 con 0x7fdd140d02f0 2024-01-19T04:55:33.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.712+0000 7fdd19cc8640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdcf806aed0 0x7fdcf806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:33.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.712+0000 7fdd19cc8640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdcf806aed0 0x7fdcf806d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdd10000f60 tx=0x7fdd10007c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:33.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.716+0000 7fdd0a7fc640 1 -- 172.21.15.31:0/2677758745 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdd04054ab0 con 0x7fdd140d02f0 2024-01-19T04:55:33.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.894+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd140cfd90 con 0x7fdd140d02f0 2024-01-19T04:55:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.895+0000 7fdd0a7fc640 1 -- 172.21.15.31:0/2677758745 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fdd040580d0 con 0x7fdd140d02f0 2024-01-19T04:55:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:33.895 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:33.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.897+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdcf806aed0 msgr2=0x7fdcf806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.897+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdcf806aed0 0x7fdcf806d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdd10000f60 tx=0x7fdd10007c00 comp rx=0 tx=0).stop 2024-01-19T04:55:33.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.897+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140d02f0 msgr2=0x7fdd141553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.897+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140d02f0 0x7fdd141553a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d800 tx=0x7fdd0400dcc0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 shutdown_connections 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdcf806aed0 0x7fdcf806d390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd14161ee0 0x7fdd141558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140d02f0 0x7fdd141553a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 --2- 172.21.15.31:0/2677758745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd140cf150 0x7fdd141537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 >> 172.21.15.31:0/2677758745 conn(0x7fdd1409c680 msgr2=0x7fdd1415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 shutdown_connections 2024-01-19T04:55:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:33.898+0000 7fdd1bf2c640 1 -- 172.21.15.31:0/2677758745 wait complete. 2024-01-19T04:55:33.908 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:33.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:34.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.012+0000 7f74dd09b640 1 Processor -- start 2024-01-19T04:55:34.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.012+0000 7f74dd09b640 1 -- start start 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74dd09b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f74d8072270 0x7f74d80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74dd09b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d80c9900 0x7f74d80ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74dd09b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d806e5d0 con 0x7f74d815b4b0 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74dd09b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d815b890 con 0x7f74d8072270 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74dd09b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d806d9c0 con 0x7f74d80c9900 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d6d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f74d815b4b0 0x7f74d8071b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59272/0 (socket says 172.21.15.31:59272) 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d6d9d640 1 -- 172.21.15.31:0/3534852429 learned_addr learned my addr 172.21.15.31:0/3534852429 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d759e640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d80c9900 0x7f74d80ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d659c640 1 --2- 172.21.15.31:0/3534852429 >> v2:172.21.15.137:3300/0 conn(0x7f74d8072270 0x7f74d80b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260651744 0 0) 0x7f74d806e5d0 con 0x7f74d815b4b0 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c0003520 con 0x7f74d815b4b0 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 162787201 0 0) 0x7f74c0003520 con 0x7f74d815b4b0 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d80c9900 msgr2=0x7f74d80ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d80c9900 0x7f74d80ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 >> v2:172.21.15.137:3300/0 conn(0x7f74d8072270 msgr2=0x7f74d80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 --2- 172.21.15.31:0/3534852429 >> v2:172.21.15.137:3300/0 conn(0x7f74d8072270 0x7f74d80b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d806dca0 con 0x7f74d815b4b0 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f74d0001670 con 0x7f74d815b4b0 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 >> v1:172.21.15.31:6789/0 conn(0x7f74d815b4b0 legacy=0x7f74d8071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c0003970 0x7f74c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.013+0000 7f74d5d9b640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74c00047c0 0x7f74c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d5d9b640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0005180 0x7f74c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d815b890 con 0x7f74c0005180 2024-01-19T04:55:34.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d806d9c0 con 0x7f74c0003970 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c000bec0 con 0x7f74c00047c0 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d759e640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d80c9900 0x7f74d80ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d659c640 1 --2- 172.21.15.31:0/3534852429 >> v2:172.21.15.137:3300/0 conn(0x7f74d8072270 0x7f74d80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d759e640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0005180 0x7f74c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d6d9d640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c0003970 0x7f74c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d659c640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74c00047c0 0x7f74c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d759e640 1 -- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74c00047c0 msgr2=0x7f74c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d759e640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74c00047c0 0x7f74c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d759e640 1 -- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c0003970 msgr2=0x7f74c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d759e640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c0003970 0x7f74c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.014+0000 7f74d759e640 1 -- 172.21.15.31:0/3534852429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0003520 con 0x7f74c0005180 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74d759e640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0005180 0x7f74c000b980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e220 tx=0x7f74c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07549f8e6c43d1c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c802cce0 con 0x7f74c0005180 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c800f040 con 0x7f74c0005180 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74d5d9b640 1 -- 172.21.15.31:0/3534852429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8003bb0 con 0x7f74c0005180 2024-01-19T04:55:34.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74dd09b640 1 -- 172.21.15.31:0/3534852429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0005180 msgr2=0x7f74c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74dd09b640 1 --2- 172.21.15.31:0/3534852429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0005180 0x7f74c000b980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e220 tx=0x7f74c00057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74d759e640 1 -- 172.21.15.31:0/3534852429 reap_dead start 2024-01-19T04:55:34.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74dd09b640 1 -- 172.21.15.31:0/3534852429 shutdown_connections 2024-01-19T04:55:34.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74dd09b640 1 -- 172.21.15.31:0/3534852429 >> 172.21.15.31:0/3534852429 conn(0x7f74d80b0da0 msgr2=0x7f74d80b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:34.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74dd09b640 1 -- 172.21.15.31:0/3534852429 shutdown_connections 2024-01-19T04:55:34.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.015+0000 7f74dd09b640 1 -- 172.21.15.31:0/3534852429 wait complete. 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 Processor -- start 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 -- start start 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d80c50c0 0x7f74d809b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d809bdd0 0x7f74d8159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74d815a4b0 0x7f74d81f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d815a940 con 0x7f74d80c50c0 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d80af3c0 con 0x7f74d815a4b0 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.017+0000 7f74dd09b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d80ae7b0 con 0x7f74d809bdd0 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d80c50c0 0x7f74d809b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d80c50c0 0x7f74d809b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43320/0 (socket says 172.21.15.31:43320) 2024-01-19T04:55:34.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 -- 172.21.15.31:0/937925648 learned_addr learned my addr 172.21.15.31:0/937925648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d659c640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d809bdd0 0x7f74d8159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d759e640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74d815a4b0 0x7f74d81f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 -- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d809bdd0 msgr2=0x7f74d8159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d809bdd0 0x7f74d8159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 -- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74d815a4b0 msgr2=0x7f74d81f0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74d815a4b0 0x7f74d81f0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 -- 172.21.15.31:0/937925648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d80aea90 con 0x7f74d80c50c0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d759e640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74d815a4b0 0x7f74d81f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74d6d9d640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d80c50c0 0x7f74d809b890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74d0007060 tx=0x7f74d0007460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74bf7fe640 1 -- 172.21.15.31:0/937925648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d000f7b0 con 0x7f74d80c50c0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74bf7fe640 1 -- 172.21.15.31:0/937925648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d000f930 con 0x7f74d80c50c0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d80aed70 con 0x7f74d80c50c0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74bf7fe640 1 -- 172.21.15.31:0/937925648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d000fce0 con 0x7f74d80c50c0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.018+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d80af850 con 0x7f74d80c50c0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.019+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d8074900 con 0x7f74d80c50c0 2024-01-19T04:55:34.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.020+0000 7f74bf7fe640 1 -- 172.21.15.31:0/937925648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f74d001d460 con 0x7f74d80c50c0 2024-01-19T04:55:34.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.020+0000 7f74bf7fe640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74c406aed0 0x7f74c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.020+0000 7f74bf7fe640 1 -- 172.21.15.31:0/937925648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74d0089760 con 0x7f74d80c50c0 2024-01-19T04:55:34.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.020+0000 7f74d659c640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74c406aed0 0x7f74c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.021+0000 7f74d659c640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74c406aed0 0x7f74c406d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74d8159660 tx=0x7f74cc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.024+0000 7f74bf7fe640 1 -- 172.21.15.31:0/937925648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f74d0054630 con 0x7f74d80c50c0 2024-01-19T04:55:34.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.201+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74d80b41c0 con 0x7f74d80c50c0 2024-01-19T04:55:34.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.201+0000 7f74bf7fe640 1 -- 172.21.15.31:0/937925648 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74d0057c50 con 0x7f74d80c50c0 2024-01-19T04:55:34.201 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74c406aed0 msgr2=0x7f74c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74c406aed0 0x7f74c406d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74d8159660 tx=0x7f74cc008040 comp rx=0 tx=0).stop 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d80c50c0 msgr2=0x7f74d809b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d80c50c0 0x7f74d809b890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74d0007060 tx=0x7f74d0007460 comp rx=0 tx=0).stop 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 shutdown_connections 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74c406aed0 0x7f74c406d390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74d815a4b0 0x7f74d81f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74d809bdd0 0x7f74d8159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 --2- 172.21.15.31:0/937925648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d80c50c0 0x7f74d809b890 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 >> 172.21.15.31:0/937925648 conn(0x7f74d80b0da0 msgr2=0x7f74d8071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 shutdown_connections 2024-01-19T04:55:34.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.203+0000 7f74dd09b640 1 -- 172.21.15.31:0/937925648 wait complete. 2024-01-19T04:55:34.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.317+0000 7efe0c410640 1 Processor -- start 2024-01-19T04:55:34.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.317+0000 7efe0c410640 1 -- start start 2024-01-19T04:55:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.318+0000 7efe0c410640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040cf6f0 0x7efe040add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.318+0000 7efe0c410640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efe040b1d60 0x7efe040b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.318+0000 7efe0c410640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040b0a70 con 0x7efe040cfad0 2024-01-19T04:55:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.318+0000 7efe0c410640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe040b18d0 con 0x7efe040b1d60 2024-01-19T04:55:34.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.318+0000 7efe0c410640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040b0cc0 con 0x7efe040cf6f0 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.318+0000 7efe0a1ac640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040cf6f0 0x7efe040add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.318+0000 7efe0a9ad640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efe040b1d60 0x7efe040b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a1ac640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040cf6f0 0x7efe040add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36612/0 (socket says 172.21.15.31:36612) 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a1ac640 1 -- 172.21.15.31:0/2718841481 learned_addr learned my addr 172.21.15.31:0/2718841481 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a9ad640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efe040b1d60 0x7efe040b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56422/0 (socket says 172.21.15.31:56422) 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe099ab640 1 --1- 172.21.15.31:0/2718841481 >> v1:172.21.15.137:6789/0 conn(0x7efe040cfad0 0x7efe040cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:42968/0 (socket says 172.21.15.31:42968) 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a1ac640 1 -- 172.21.15.31:0/2718841481 >> v2:172.21.15.137:3300/0 conn(0x7efe040b1d60 msgr2=0x7efe040b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a1ac640 1 --2- 172.21.15.31:0/2718841481 >> v2:172.21.15.137:3300/0 conn(0x7efe040b1d60 0x7efe040b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a1ac640 1 -- 172.21.15.31:0/2718841481 >> v1:172.21.15.137:6789/0 conn(0x7efe040cfad0 legacy=0x7efe040cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a1ac640 1 -- 172.21.15.31:0/2718841481 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040b0fa0 con 0x7efe040cf6f0 2024-01-19T04:55:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.319+0000 7efe0a9ad640 1 --2- 172.21.15.31:0/2718841481 >> v2:172.21.15.137:3300/0 conn(0x7efe040b1d60 0x7efe040b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0a1ac640 1 --2- 172.21.15.31:0/2718841481 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040cf6f0 0x7efe040add00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009870 tx=0x7efdfc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b4c04f24a951ef server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe091aa640 1 -- 172.21.15.31:0/2718841481 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc030840 con 0x7efe040cf6f0 2024-01-19T04:55:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe091aa640 1 -- 172.21.15.31:0/2718841481 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc0309a0 con 0x7efe040cf6f0 2024-01-19T04:55:34.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe091aa640 1 -- 172.21.15.31:0/2718841481 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc030d30 con 0x7efe040cf6f0 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 -- 172.21.15.31:0/2718841481 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040cf6f0 msgr2=0x7efe040add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 --2- 172.21.15.31:0/2718841481 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040cf6f0 0x7efe040add00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009870 tx=0x7efdfc02b980 comp rx=0 tx=0).stop 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 -- 172.21.15.31:0/2718841481 shutdown_connections 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 --2- 172.21.15.31:0/2718841481 >> v2:172.21.15.137:3300/0 conn(0x7efe040b1d60 0x7efe040b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 --2- 172.21.15.31:0/2718841481 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040cf6f0 0x7efe040add00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 -- 172.21.15.31:0/2718841481 >> 172.21.15.31:0/2718841481 conn(0x7efe0408afe0 msgr2=0x7efe040d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 -- 172.21.15.31:0/2718841481 shutdown_connections 2024-01-19T04:55:34.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.320+0000 7efe0c410640 1 -- 172.21.15.31:0/2718841481 wait complete. 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.321+0000 7efe0c410640 1 Processor -- start 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0c410640 1 -- start start 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0c410640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040b1d60 0x7efe040c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0c410640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040cf6f0 0x7efe040c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0c410640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cfad0 0x7efe040af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0c410640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0415dd30 con 0x7efe040cfad0 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0c410640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0415d120 con 0x7efe040cf6f0 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0c410640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0415d400 con 0x7efe040b1d60 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0a9ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cfad0 0x7efe040af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0a9ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cfad0 0x7efe040af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43332/0 (socket says 172.21.15.31:43332) 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0a9ad640 1 -- 172.21.15.31:0/1564326520 learned_addr learned my addr 172.21.15.31:0/1564326520 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe099ab640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040cf6f0 0x7efe040c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.322+0000 7efe0a1ac640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040b1d60 0x7efe040c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0a9ad640 1 -- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040b1d60 msgr2=0x7efe040c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0a9ad640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040b1d60 0x7efe040c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0a9ad640 1 -- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040cf6f0 msgr2=0x7efe040c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0a9ad640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040cf6f0 0x7efe040c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0a9ad640 1 -- 172.21.15.31:0/1564326520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009a70 con 0x7efe040cfad0 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0a1ac640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040b1d60 0x7efe040c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0a9ad640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cfad0 0x7efe040af070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efdf400efc0 tx=0x7efdf400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe02ffd640 1 -- 172.21.15.31:0/1564326520 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4009cd0 con 0x7efe040cfad0 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe02ffd640 1 -- 172.21.15.31:0/1564326520 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400f040 con 0x7efe040cfad0 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc0094d0 con 0x7efe040cfad0 2024-01-19T04:55:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe02ffd640 1 -- 172.21.15.31:0/1564326520 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4003990 con 0x7efe040cfad0 2024-01-19T04:55:34.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.323+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0415d9c0 con 0x7efe040cfad0 2024-01-19T04:55:34.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.324+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0415dee0 con 0x7efe040cfad0 2024-01-19T04:55:34.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.325+0000 7efe02ffd640 1 -- 172.21.15.31:0/1564326520 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efdf4014480 con 0x7efe040cfad0 2024-01-19T04:55:34.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.325+0000 7efe02ffd640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006aed0 0x7efdf006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.325+0000 7efe02ffd640 1 -- 172.21.15.31:0/1564326520 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdf408a0b0 con 0x7efe040cfad0 2024-01-19T04:55:34.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.325+0000 7efe0a1ac640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006aed0 0x7efdf006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.326+0000 7efe0a1ac640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006aed0 0x7efdf006d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdfc036000 tx=0x7efdfc03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.329+0000 7efe02ffd640 1 -- 172.21.15.31:0/1564326520 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efe0415dee0 con 0x7efe040cfad0 2024-01-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.509+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe04074900 con 0x7efe040cfad0 2024-01-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.509+0000 7efe02ffd640 1 -- 172.21.15.31:0/1564326520 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7efdf4058aa0 con 0x7efe040cfad0 2024-01-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:34.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006aed0 msgr2=0x7efdf006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006aed0 0x7efdf006d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdfc036000 tx=0x7efdfc03b040 comp rx=0 tx=0).stop 2024-01-19T04:55:34.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cfad0 msgr2=0x7efe040af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cfad0 0x7efe040af070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efdf400efc0 tx=0x7efdf400c490 comp rx=0 tx=0).stop 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 shutdown_connections 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006aed0 0x7efdf006d390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cfad0 0x7efe040af070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040cf6f0 0x7efe040c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 --2- 172.21.15.31:0/1564326520 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040b1d60 0x7efe040c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.511+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 >> 172.21.15.31:0/1564326520 conn(0x7efe0408afe0 msgr2=0x7efe040ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.512+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 shutdown_connections 2024-01-19T04:55:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.512+0000 7efe0c410640 1 -- 172.21.15.31:0/1564326520 wait complete. 2024-01-19T04:55:34.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:34.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.626+0000 7f2eebba4640 1 Processor -- start 2024-01-19T04:55:34.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.626+0000 7f2eebba4640 1 -- start start 2024-01-19T04:55:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2eebba4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 0x7f2ee4161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2eebba4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40cf9f0 con 0x7f2ee415d780 2024-01-19T04:55:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2eebba4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40cfcd0 con 0x7f2ee415d3a0 2024-01-19T04:55:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2eebba4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cffb0 con 0x7f2ee40cf570 2024-01-19T04:55:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2eea141640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2ee415d780 0x7f2ee4167480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59278/0 (socket says 172.21.15.31:59278) 2024-01-19T04:55:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2eea141640 1 -- 172.21.15.31:0/3781401712 learned_addr learned my addr 172.21.15.31:0/3781401712 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee913f640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 0x7f2ee4161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177123020 0 0) 0x7f2ee40cf9f0 con 0x7f2ee415d780 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed0003520 con 0x7f2ee415d780 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1231248617 0 0) 0x7f2ed0003520 con 0x7f2ee415d780 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 msgr2=0x7f2ee4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 0x7f2ee4161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 >> v1:172.21.15.137:6789/0 conn(0x7f2ee415d3a0 legacy=0x7f2ee406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40d0290 con 0x7f2ee415d780 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f2edc0019f0 con 0x7f2ee415d780 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 >> v1:172.21.15.31:6789/0 conn(0x7f2ee415d780 legacy=0x7f2ee4167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed00039a0 0x7f2ed0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0004350 0x7f2ed0004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 0x7f2ed0005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cffb0 con 0x7f2ed00039a0 2024-01-19T04:55:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed000b9b0 con 0x7f2ed0004350 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed000bb70 con 0x7f2ee40cf570 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.627+0000 7f2ee9940640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed00039a0 0x7f2ed0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee913f640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0004350 0x7f2ed0004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2eea141640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 0x7f2ed0005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee9940640 1 -- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 msgr2=0x7f2ed0005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee9940640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 0x7f2ed0005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee9940640 1 -- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0004350 msgr2=0x7f2ed0004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee9940640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0004350 0x7f2ed0004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee9940640 1 -- 172.21.15.31:0/3781401712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0003520 con 0x7f2ed00039a0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2eea141640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee40cf570 0x7f2ed0005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee9940640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed00039a0 0x7f2ed0003d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0001930 tx=0x7f2ed00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b72275879235f8a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.628+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0036070 con 0x7f2ed00039a0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0006820 con 0x7f2ed00039a0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2ee893e640 1 -- 172.21.15.31:0/3781401712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0006bb0 con 0x7f2ed00039a0 2024-01-19T04:55:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2eebba4640 1 -- 172.21.15.31:0/3781401712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed00039a0 msgr2=0x7f2ed0003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2eebba4640 1 --2- 172.21.15.31:0/3781401712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed00039a0 0x7f2ed0003d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0001930 tx=0x7f2ed00057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2eea141640 1 -- 172.21.15.31:0/3781401712 reap_dead start 2024-01-19T04:55:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2eebba4640 1 -- 172.21.15.31:0/3781401712 shutdown_connections 2024-01-19T04:55:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2eebba4640 1 -- 172.21.15.31:0/3781401712 >> 172.21.15.31:0/3781401712 conn(0x7f2ee40d1100 msgr2=0x7f2ee40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2eebba4640 1 -- 172.21.15.31:0/3781401712 shutdown_connections 2024-01-19T04:55:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.629+0000 7f2eebba4640 1 -- 172.21.15.31:0/3781401712 wait complete. 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.630+0000 7f2eebba4640 1 Processor -- start 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.630+0000 7f2eebba4640 1 -- start start 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eebba4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed00039a0 0x7f2ee4156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eebba4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee4158e70 0x7f2ee41592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eebba4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee409db80 0x7f2ee409e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eebba4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee4159940 con 0x7f2ee409db80 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eebba4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee409be60 con 0x7f2ed00039a0 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eebba4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee409e550 con 0x7f2ee4158e70 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee409db80 0x7f2ee409e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee409db80 0x7f2ee409e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43354/0 (socket says 172.21.15.31:43354) 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 -- 172.21.15.31:0/3030052858 learned_addr learned my addr 172.21.15.31:0/3030052858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2ee913f640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee4158e70 0x7f2ee41592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2ee9940640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed00039a0 0x7f2ee4156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 -- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee4158e70 msgr2=0x7f2ee41592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee4158e70 0x7f2ee41592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 -- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed00039a0 msgr2=0x7f2ee4156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed00039a0 0x7f2ee4156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2eea141640 1 -- 172.21.15.31:0/3030052858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee409b570 con 0x7f2ee409db80 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.631+0000 7f2ee913f640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee4158e70 0x7f2ee41592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.632+0000 7f2eea141640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee409db80 0x7f2ee409e010 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b71b0 tx=0x7f2edc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.632+0000 7f2eda7fc640 1 -- 172.21.15.31:0/3030052858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00eb90 con 0x7f2ee409db80 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.632+0000 7f2eda7fc640 1 -- 172.21.15.31:0/3030052858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc00ecf0 con 0x7f2ee409db80 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.632+0000 7f2eda7fc640 1 -- 172.21.15.31:0/3030052858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc013620 con 0x7f2ee409db80 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.632+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee409b810 con 0x7f2ee409db80 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.632+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee4156980 con 0x7f2ee409db80 2024-01-19T04:55:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.633+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee409c010 con 0x7f2ee409db80 2024-01-19T04:55:34.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.633+0000 7f2eda7fc640 1 -- 172.21.15.31:0/3030052858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2edc013780 con 0x7f2ee409db80 2024-01-19T04:55:34.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.633+0000 7f2eda7fc640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2ec806aed0 0x7f2ec806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.634+0000 7f2eda7fc640 1 -- 172.21.15.31:0/3030052858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2edc089970 con 0x7f2ee409db80 2024-01-19T04:55:34.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.634+0000 7f2ee9940640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2ec806aed0 0x7f2ec806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.634+0000 7f2ee9940640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2ec806aed0 0x7f2ec806d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0035000 tx=0x7f2ee0009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.638+0000 7f2eda7fc640 1 -- 172.21.15.31:0/3030052858 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2ee409c010 con 0x7f2ee409db80 2024-01-19T04:55:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.815+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ee409c010 con 0x7f2ee409db80 2024-01-19T04:55:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.815+0000 7f2eda7fc640 1 -- 172.21.15.31:0/3030052858 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ee409c010 con 0x7f2ee409db80 2024-01-19T04:55:34.815 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2ec806aed0 msgr2=0x7f2ec806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2ec806aed0 0x7f2ec806d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0035000 tx=0x7f2ee0009030 comp rx=0 tx=0).stop 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee409db80 msgr2=0x7f2ee409e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee409db80 0x7f2ee409e010 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b71b0 tx=0x7f2edc00e530 comp rx=0 tx=0).stop 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 shutdown_connections 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2ec806aed0 0x7f2ec806d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee409db80 0x7f2ee409e010 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ee4158e70 0x7f2ee41592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 --2- 172.21.15.31:0/3030052858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed00039a0 0x7f2ee4156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 >> 172.21.15.31:0/3030052858 conn(0x7f2ee40d1100 msgr2=0x7f2ee415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 shutdown_connections 2024-01-19T04:55:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.817+0000 7f2eebba4640 1 -- 172.21.15.31:0/3030052858 wait complete. 2024-01-19T04:55:34.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:55:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.928+0000 7f919c5f5640 1 Processor -- start 2024-01-19T04:55:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.928+0000 7f919c5f5640 1 -- start start 2024-01-19T04:55:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919c5f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91940b3450 0x7f919415e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919c5f5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91940cb8b0 con 0x7f91941580f0 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919c5f5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919415f740 con 0x7f919415de70 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919c5f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919406c300 con 0x7f91940b3450 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919ab92640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91940b3450 0x7f919415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919a391640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f91941580f0 0x7f919415f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59292/0 (socket says 172.21.15.31:59292) 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919ab92640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91940b3450 0x7f919415e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43366/0 (socket says 172.21.15.31:43366) 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919a391640 1 -- 172.21.15.31:0/1474115065 learned_addr learned my addr 172.21.15.31:0/1474115065 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919ab92640 1 -- 172.21.15.31:0/1474115065 >> v1:172.21.15.137:6789/0 conn(0x7f919415de70 legacy=0x7f91940b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919ab92640 1 -- 172.21.15.31:0/1474115065 >> v1:172.21.15.31:6789/0 conn(0x7f91941580f0 legacy=0x7f919415f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919ab92640 1 -- 172.21.15.31:0/1474115065 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919415e960 con 0x7f91940b3450 2024-01-19T04:55:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.929+0000 7f919938f640 1 -- 172.21.15.31:0/1474115065 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137988948 0 0) 0x7f91940cb8b0 con 0x7f91941580f0 2024-01-19T04:55:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919ab92640 1 --2- 172.21.15.31:0/1474115065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91940b3450 0x7f919415e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f918c009d90 tx=0x7f918c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2b30887255a765 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919938f640 1 -- 172.21.15.31:0/1474115065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c02cd00 con 0x7f91940b3450 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919938f640 1 -- 172.21.15.31:0/1474115065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c02ce60 con 0x7f91940b3450 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919938f640 1 -- 172.21.15.31:0/1474115065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c034660 con 0x7f91940b3450 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919c5f5640 1 -- 172.21.15.31:0/1474115065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91940b3450 msgr2=0x7f919415e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919c5f5640 1 --2- 172.21.15.31:0/1474115065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91940b3450 0x7f919415e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f918c009d90 tx=0x7f918c02bf50 comp rx=0 tx=0).stop 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919c5f5640 1 -- 172.21.15.31:0/1474115065 shutdown_connections 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919c5f5640 1 --2- 172.21.15.31:0/1474115065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91940b3450 0x7f919415e320 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.930+0000 7f919c5f5640 1 -- 172.21.15.31:0/1474115065 >> 172.21.15.31:0/1474115065 conn(0x7f919406abe0 msgr2=0x7f9194066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.931+0000 7f919c5f5640 1 -- 172.21.15.31:0/1474115065 shutdown_connections 2024-01-19T04:55:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.931+0000 7f919c5f5640 1 -- 172.21.15.31:0/1474115065 wait complete. 2024-01-19T04:55:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 Processor -- start 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 -- start start 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f91940b3450 0x7f91940c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941580f0 0x7f91941477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f919415de70 0x7f91940d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919415b230 con 0x7f919415de70 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91941f2f20 con 0x7f91941580f0 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919c5f5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9194147ce0 con 0x7f91940b3450 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919ab92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f919415de70 0x7f91940d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919ab92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f919415de70 0x7f91940d1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43372/0 (socket says 172.21.15.31:43372) 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.932+0000 7f919ab92640 1 -- 172.21.15.31:0/2428088259 learned_addr learned my addr 172.21.15.31:0/2428088259 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f9199b90640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941580f0 0x7f91941477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919a391640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f91940b3450 0x7f91940c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919ab92640 1 -- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f91940b3450 msgr2=0x7f91940c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919ab92640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f91940b3450 0x7f91940c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919ab92640 1 -- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941580f0 msgr2=0x7f91941477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919ab92640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941580f0 0x7f91941477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919ab92640 1 -- 172.21.15.31:0/2428088259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c009a70 con 0x7f919415de70 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f9199b90640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941580f0 0x7f91941477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919ab92640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f919415de70 0x7f91940d1710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f918c037040 tx=0x7f918c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f918affd640 1 -- 172.21.15.31:0/2428088259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c003b90 con 0x7f919415de70 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f918affd640 1 -- 172.21.15.31:0/2428088259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c003cf0 con 0x7f919415de70 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919415a620 con 0x7f919415de70 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f918affd640 1 -- 172.21.15.31:0/2428088259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c004270 con 0x7f919415de70 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.933+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919415abe0 con 0x7f919415de70 2024-01-19T04:55:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.934+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919415b3e0 con 0x7f919415de70 2024-01-19T04:55:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.935+0000 7f918affd640 1 -- 172.21.15.31:0/2428088259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f918c03b040 con 0x7f919415de70 2024-01-19T04:55:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.935+0000 7f918affd640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f917806aed0 0x7f917806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.935+0000 7f918affd640 1 -- 172.21.15.31:0/2428088259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f918c0ab8a0 con 0x7f919415de70 2024-01-19T04:55:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.935+0000 7f919a391640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f917806aed0 0x7f917806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.936+0000 7f919a391640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f917806aed0 0x7f917806d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9190000f60 tx=0x7f9190007b90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:34.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:34.939+0000 7f918affd640 1 -- 172.21.15.31:0/2428088259 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f919415b3e0 con 0x7f919415de70 2024-01-19T04:55:35.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.156+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91940696e0 con 0x7f919415de70 2024-01-19T04:55:35.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.157+0000 7f918affd640 1 -- 172.21.15.31:0/2428088259 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f918c039090 con 0x7f919415de70 2024-01-19T04:55:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:35.157 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:55:35.157 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:55:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f917806aed0 msgr2=0x7f917806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f917806aed0 0x7f917806d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9190000f60 tx=0x7f9190007b90 comp rx=0 tx=0).stop 2024-01-19T04:55:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f919415de70 msgr2=0x7f91940d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f919415de70 0x7f91940d1710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f918c037040 tx=0x7f918c004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 shutdown_connections 2024-01-19T04:55:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f917806aed0 0x7f917806d390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f919415de70 0x7f91940d1710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941580f0 0x7f91941477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 --2- 172.21.15.31:0/2428088259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f91940b3450 0x7f91940c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 >> 172.21.15.31:0/2428088259 conn(0x7f919406abe0 msgr2=0x7f919415c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 shutdown_connections 2024-01-19T04:55:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.159+0000 7f919c5f5640 1 -- 172.21.15.31:0/2428088259 wait complete. 2024-01-19T04:55:35.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:35.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.271+0000 7f551a289640 1 Processor -- start 2024-01-19T04:55:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.271+0000 7f551a289640 1 -- start start 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f551a289640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5514160bb0 0x7f55140b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f551a289640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f55140cfd10 0x7f55140d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f551a289640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55140d06b0 con 0x7f55140d0ef0 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f551a289640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55140b66b0 con 0x7f55140cfd10 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f551a289640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140d1b90 con 0x7f5514160bb0 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f5518826640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f55140d0ef0 0x7f55140d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59298/0 (socket says 172.21.15.31:59298) 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f5518826640 1 -- 172.21.15.31:0/1505269678 learned_addr learned my addr 172.21.15.31:0/1505269678 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f55137fe640 1 --2- 172.21.15.31:0/1505269678 >> v2:172.21.15.137:3300/0 conn(0x7f55140cfd10 0x7f55140d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f5513fff640 1 --2- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5514160bb0 0x7f55140b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f5512ffd640 1 -- 172.21.15.31:0/1505269678 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576717382 0 0) 0x7f55140d06b0 con 0x7f55140d0ef0 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f5512ffd640 1 -- 172.21.15.31:0/1505269678 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54fc003520 con 0x7f55140d0ef0 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f55137fe640 1 -- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5514160bb0 msgr2=0x7f55140b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f55137fe640 1 --2- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5514160bb0 0x7f55140b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f55137fe640 1 -- 172.21.15.31:0/1505269678 >> v1:172.21.15.31:6789/0 conn(0x7f55140d0ef0 legacy=0x7f55140d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f5512ffd640 1 -- 172.21.15.31:0/1505269678 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3800525745 0 0) 0x7f54fc003520 con 0x7f55140d0ef0 2024-01-19T04:55:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.272+0000 7f55137fe640 1 -- 172.21.15.31:0/1505269678 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140d1e70 con 0x7f55140cfd10 2024-01-19T04:55:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.273+0000 7f55137fe640 1 --2- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55140cfd10 0x7f55140d0170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f55080097a0 tx=0x7f550802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=438297edd77fbeff server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.273+0000 7f5512ffd640 1 -- 172.21.15.31:0/1505269678 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c0019f0 con 0x7f55140cfd10 2024-01-19T04:55:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.273+0000 7f5512ffd640 1 -- 172.21.15.31:0/1505269678 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550802e050 con 0x7f55140cfd10 2024-01-19T04:55:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.273+0000 7f5512ffd640 1 -- 172.21.15.31:0/1505269678 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508030710 con 0x7f55140cfd10 2024-01-19T04:55:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 -- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55140cfd10 msgr2=0x7f55140d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 --2- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55140cfd10 0x7f55140d0170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f55080097a0 tx=0x7f550802b790 comp rx=0 tx=0).stop 2024-01-19T04:55:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 -- 172.21.15.31:0/1505269678 shutdown_connections 2024-01-19T04:55:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 --2- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55140cfd10 0x7f55140d0170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 --2- 172.21.15.31:0/1505269678 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5514160bb0 0x7f55140b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 -- 172.21.15.31:0/1505269678 >> 172.21.15.31:0/1505269678 conn(0x7f551409c600 msgr2=0x7f55140d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 -- 172.21.15.31:0/1505269678 shutdown_connections 2024-01-19T04:55:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.274+0000 7f551a289640 1 -- 172.21.15.31:0/1505269678 wait complete. 2024-01-19T04:55:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.275+0000 7f551a289640 1 Processor -- start 2024-01-19T04:55:35.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.275+0000 7f551a289640 1 -- start start 2024-01-19T04:55:35.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f551a289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55140cfd10 0x7f551415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f551a289640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55140d0ef0 0x7f55140cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f551a289640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5514160bb0 0x7f5514158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f551a289640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551415a620 con 0x7f55140cfd10 2024-01-19T04:55:35.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f551a289640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514158fe0 con 0x7f5514160bb0 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f551a289640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140cb500 con 0x7f55140d0ef0 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55140cfd10 0x7f551415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55140cfd10 0x7f551415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43388/0 (socket says 172.21.15.31:43388) 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 -- 172.21.15.31:0/102920110 learned_addr learned my addr 172.21.15.31:0/102920110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f55137fe640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55140d0ef0 0x7f55140cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5518826640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5514160bb0 0x7f5514158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 -- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55140d0ef0 msgr2=0x7f55140cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55140d0ef0 0x7f55140cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 -- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5514160bb0 msgr2=0x7f5514158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5514160bb0 0x7f5514158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5513fff640 1 -- 172.21.15.31:0/102920110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5508009400 con 0x7f55140cfd10 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.276+0000 7f5518826640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5514160bb0 0x7f5514158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.277+0000 7f5513fff640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55140cfd10 0x7f551415aff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f550400e9b0 tx=0x7f550400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.277+0000 7f5510ff9640 1 -- 172.21.15.31:0/102920110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504009670 con 0x7f55140cfd10 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.277+0000 7f5510ff9640 1 -- 172.21.15.31:0/102920110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400c9a0 con 0x7f55140cfd10 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.277+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55140cb7e0 con 0x7f55140cfd10 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.277+0000 7f5510ff9640 1 -- 172.21.15.31:0/102920110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400cd50 con 0x7f55140cfd10 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.277+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55140cbda0 con 0x7f55140cfd10 2024-01-19T04:55:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.278+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55140696e0 con 0x7f55140cfd10 2024-01-19T04:55:35.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.278+0000 7f5510ff9640 1 -- 172.21.15.31:0/102920110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5504013470 con 0x7f55140cfd10 2024-01-19T04:55:35.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.279+0000 7f5510ff9640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f550006aed0 0x7f550006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.279+0000 7f5510ff9640 1 -- 172.21.15.31:0/102920110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55040898c0 con 0x7f55140cfd10 2024-01-19T04:55:35.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.279+0000 7f55137fe640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f550006aed0 0x7f550006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.279+0000 7f55137fe640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f550006aed0 0x7f550006d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5508034000 tx=0x7f550802f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:35.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.283+0000 7f5510ff9640 1 -- 172.21.15.31:0/102920110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5504054780 con 0x7f55140cfd10 2024-01-19T04:55:35.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.459+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f551408d3b0 con 0x7f55140cfd10 2024-01-19T04:55:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.460+0000 7f5510ff9640 1 -- 172.21.15.31:0/102920110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f5504017090 con 0x7f55140cfd10 2024-01-19T04:55:35.460 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:35.460 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f550006aed0 msgr2=0x7f550006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f550006aed0 0x7f550006d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5508034000 tx=0x7f550802f000 comp rx=0 tx=0).stop 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55140cfd10 msgr2=0x7f551415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55140cfd10 0x7f551415aff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f550400e9b0 tx=0x7f550400ee70 comp rx=0 tx=0).stop 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 shutdown_connections 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f550006aed0 0x7f550006d390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5514160bb0 0x7f5514158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55140d0ef0 0x7f55140cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 --2- 172.21.15.31:0/102920110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55140cfd10 0x7f551415aff0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 >> 172.21.15.31:0/102920110 conn(0x7f551409c600 msgr2=0x7f551406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:35.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 shutdown_connections 2024-01-19T04:55:35.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.462+0000 7f551a289640 1 -- 172.21.15.31:0/102920110 wait complete. 2024-01-19T04:55:35.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:55:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.576+0000 7f09a391f640 1 Processor -- start 2024-01-19T04:55:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.576+0000 7f09a391f640 1 -- start start 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a391f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f099c15d8a0 0x7f099c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a391f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c15eb70 0x7f099c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a391f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c06bf30 con 0x7f099c0cca00 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a391f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c0cb9e0 con 0x7f099c15d8a0 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a391f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0cbcc0 con 0x7f099c15eb70 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a16bb640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f099c15d8a0 0x7f099c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a0eba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c15eb70 0x7f099c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a1ebc640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f099c0cca00 0x7f099c15f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:42986/0 (socket says 172.21.15.31:42986) 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a1ebc640 1 -- 172.21.15.31:0/758354153 learned_addr learned my addr 172.21.15.31:0/758354153 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a0eba640 1 -- 172.21.15.31:0/758354153 >> v2:172.21.15.137:3300/0 conn(0x7f099c15d8a0 msgr2=0x7f099c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a0eba640 1 --2- 172.21.15.31:0/758354153 >> v2:172.21.15.137:3300/0 conn(0x7f099c15d8a0 0x7f099c06b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a0eba640 1 -- 172.21.15.31:0/758354153 >> v1:172.21.15.137:6789/0 conn(0x7f099c0cca00 legacy=0x7f099c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.577+0000 7f09a0eba640 1 -- 172.21.15.31:0/758354153 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c0cbfa0 con 0x7f099c15eb70 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.578+0000 7f09a0eba640 1 --2- 172.21.15.31:0/758354153 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c15eb70 0x7f099c0cc4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f098c0097a0 tx=0x7f098c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991e58b1ca2b220f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.578+0000 7f0993fff640 1 -- 172.21.15.31:0/758354153 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c02e070 con 0x7f099c15eb70 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.578+0000 7f0993fff640 1 -- 172.21.15.31:0/758354153 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c0315a0 con 0x7f099c15eb70 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.578+0000 7f0993fff640 1 -- 172.21.15.31:0/758354153 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c031930 con 0x7f099c15eb70 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 -- 172.21.15.31:0/758354153 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c15eb70 msgr2=0x7f099c0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 --2- 172.21.15.31:0/758354153 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c15eb70 0x7f099c0cc4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f098c0097a0 tx=0x7f098c02b840 comp rx=0 tx=0).stop 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 -- 172.21.15.31:0/758354153 shutdown_connections 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 --2- 172.21.15.31:0/758354153 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c15eb70 0x7f099c0cc4c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 --2- 172.21.15.31:0/758354153 >> v2:172.21.15.137:3300/0 conn(0x7f099c15d8a0 0x7f099c06b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 -- 172.21.15.31:0/758354153 >> 172.21.15.31:0/758354153 conn(0x7f099c06abe0 msgr2=0x7f099c066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 -- 172.21.15.31:0/758354153 shutdown_connections 2024-01-19T04:55:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.579+0000 7f09a391f640 1 -- 172.21.15.31:0/758354153 wait complete. 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.580+0000 7f09a391f640 1 Processor -- start 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.580+0000 7f09a391f640 1 -- start start 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a391f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c0cca00 0x7f099c0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a391f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f099c15d8a0 0x7f099c1f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a391f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099c15eb70 0x7f099c0b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a391f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0c7cb0 con 0x7f099c15eb70 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a391f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0b6e80 con 0x7f099c15d8a0 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a391f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0b44d0 con 0x7f099c0cca00 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099c15eb70 0x7f099c0b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099c15eb70 0x7f099c0b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43396/0 (socket says 172.21.15.31:43396) 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 -- 172.21.15.31:0/1201416820 learned_addr learned my addr 172.21.15.31:0/1201416820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a0eba640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f099c15d8a0 0x7f099c1f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a16bb640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c0cca00 0x7f099c0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 -- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c0cca00 msgr2=0x7f099c0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c0cca00 0x7f099c0b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 -- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f099c15d8a0 msgr2=0x7f099c1f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f099c15d8a0 0x7f099c1f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.581+0000 7f09a1ebc640 1 -- 172.21.15.31:0/1201416820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009400 con 0x7f099c15eb70 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.582+0000 7f09a1ebc640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099c15eb70 0x7f099c0b5a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f099400ca80 tx=0x7f099400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.582+0000 7f0991ffb640 1 -- 172.21.15.31:0/1201416820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099400ac30 con 0x7f099c15eb70 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.582+0000 7f0991ffb640 1 -- 172.21.15.31:0/1201416820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099400ad90 con 0x7f099c15eb70 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.582+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c0cfc10 con 0x7f099c15eb70 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.582+0000 7f0991ffb640 1 -- 172.21.15.31:0/1201416820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994012600 con 0x7f099c15eb70 2024-01-19T04:55:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.582+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c0d01d0 con 0x7f099c15eb70 2024-01-19T04:55:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.583+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c0696e0 con 0x7f099c15eb70 2024-01-19T04:55:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.583+0000 7f0991ffb640 1 -- 172.21.15.31:0/1201416820 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0994007070 con 0x7f099c15eb70 2024-01-19T04:55:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.584+0000 7f0991ffb640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f098006aed0 0x7f098006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.584+0000 7f0991ffb640 1 -- 172.21.15.31:0/1201416820 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0994089480 con 0x7f099c15eb70 2024-01-19T04:55:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.584+0000 7f09a16bb640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f098006aed0 0x7f098006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.584+0000 7f09a16bb640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f098006aed0 0x7f098006d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0998009aa0 tx=0x7f0998009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:35.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.588+0000 7f0991ffb640 1 -- 172.21.15.31:0/1201416820 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0994054370 con 0x7f099c15eb70 2024-01-19T04:55:35.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.802+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f099c0b71e0 con 0x7f099c15eb70 2024-01-19T04:55:35.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.803+0000 7f0991ffb640 1 -- 172.21.15.31:0/1201416820 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f0994017090 con 0x7f099c15eb70 2024-01-19T04:55:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:35.802 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:55:35.803 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:55:35.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f098006aed0 msgr2=0x7f098006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f098006aed0 0x7f098006d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0998009aa0 tx=0x7f0998009190 comp rx=0 tx=0).stop 2024-01-19T04:55:35.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099c15eb70 msgr2=0x7f099c0b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:35.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099c15eb70 0x7f099c0b5a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f099400ca80 tx=0x7f099400cf40 comp rx=0 tx=0).stop 2024-01-19T04:55:35.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 shutdown_connections 2024-01-19T04:55:35.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f098006aed0 0x7f098006d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099c15eb70 0x7f099c0b5a40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f099c15d8a0 0x7f099c1f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 --2- 172.21.15.31:0/1201416820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f099c0cca00 0x7f099c0b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:35.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.804+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 >> 172.21.15.31:0/1201416820 conn(0x7f099c06abe0 msgr2=0x7f099c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:35.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.805+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 shutdown_connections 2024-01-19T04:55:35.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:35.805+0000 7f09a391f640 1 -- 172.21.15.31:0/1201416820 wait complete. 2024-01-19T04:55:36.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-19T04:55:18.877117+0000', 'modified': '2024-01-19T04:55:20.870670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6833/2576238027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6832', 'nonce': 2576238027}, {'type': 'v1', 'addr': '172.21.15.31:6833', 'nonce': 2576238027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-19T04:55:36.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-19T04:55:36.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-19T04:55:36.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.322+0000 7fcf777b0640 1 Processor -- start 2024-01-19T04:55:36.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.323+0000 7fcf777b0640 1 -- start start 2024-01-19T04:55:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.323+0000 7fcf777b0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcf7015d3a0 0x7fcf7006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.323+0000 7fcf777b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf70167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.323+0000 7fcf777b0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf700cf9f0 con 0x7fcf700cf570 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.323+0000 7fcf777b0640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700cfcd0 con 0x7fcf7015d3a0 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.323+0000 7fcf777b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700cffb0 con 0x7fcf70161860 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf7554c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcf7015d3a0 0x7fcf7006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf75d4d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf70167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf75d4d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf70167480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36720/0 (socket says 172.21.15.31:36720) 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1475330145 learned_addr learned my addr 172.21.15.31:0/1475330145 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf7554c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcf7015d3a0 0x7fcf7006c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:56516/0 (socket says 172.21.15.31:56516) 2024-01-19T04:55:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1475330145 >> v2:172.21.15.137:3300/0 conn(0x7fcf7015d3a0 msgr2=0x7fcf7006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/1475330145 >> v2:172.21.15.137:3300/0 conn(0x7fcf7015d3a0 0x7fcf7006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1475330145 >> v1:172.21.15.137:6789/0 conn(0x7fcf700cf570 legacy=0x7fcf70161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1475330145 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700d0290 con 0x7fcf70161860 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.324+0000 7fcf7554c640 1 --2- 172.21.15.31:0/1475330145 >> v2:172.21.15.137:3300/0 conn(0x7fcf7015d3a0 0x7fcf7006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.325+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/1475330145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf70167480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009870 tx=0x7fcf6002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4005956b5dd17b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.325+0000 7fcf6ffff640 1 -- 172.21.15.31:0/1475330145 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60030840 con 0x7fcf70161860 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.325+0000 7fcf6ffff640 1 -- 172.21.15.31:0/1475330145 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf600309a0 con 0x7fcf70161860 2024-01-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.325+0000 7fcf6ffff640 1 -- 172.21.15.31:0/1475330145 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60030d30 con 0x7fcf70161860 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 -- 172.21.15.31:0/1475330145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 msgr2=0x7fcf70167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 --2- 172.21.15.31:0/1475330145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf70167480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009870 tx=0x7fcf6002b980 comp rx=0 tx=0).stop 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 -- 172.21.15.31:0/1475330145 shutdown_connections 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 --2- 172.21.15.31:0/1475330145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf70167480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 --2- 172.21.15.31:0/1475330145 >> v2:172.21.15.137:3300/0 conn(0x7fcf7015d3a0 0x7fcf7006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 -- 172.21.15.31:0/1475330145 >> 172.21.15.31:0/1475330145 conn(0x7fcf700d1100 msgr2=0x7fcf700d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 -- 172.21.15.31:0/1475330145 shutdown_connections 2024-01-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.326+0000 7fcf777b0640 1 -- 172.21.15.31:0/1475330145 wait complete. 2024-01-19T04:55:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.327+0000 7fcf777b0640 1 Processor -- start 2024-01-19T04:55:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.327+0000 7fcf777b0640 1 -- start start 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf777b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf700cf570 0x7fcf700b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf777b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf7015d3a0 0x7fcf701e7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf777b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf700c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf777b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700c57a0 con 0x7fcf700cf570 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf777b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700c4c00 con 0x7fcf7015d3a0 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf777b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700c4e70 con 0x7fcf70161860 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf700cf570 0x7fcf700b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf700cf570 0x7fcf700b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43408/0 (socket says 172.21.15.31:43408) 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 -- 172.21.15.31:0/97838164 learned_addr learned my addr 172.21.15.31:0/97838164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf7015d3a0 0x7fcf701e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf700c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 -- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 msgr2=0x7fcf700c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf700c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 -- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf7015d3a0 msgr2=0x7fcf701e7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf7015d3a0 0x7fcf701e7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 -- 172.21.15.31:0/97838164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68009a70 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf7554c640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf700cf570 0x7fcf700b5c20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800efc0 tx=0x7fcf6800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf6dffb640 1 -- 172.21.15.31:0/97838164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68014680 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf6dffb640 1 -- 172.21.15.31:0/97838164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6800f040 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf600094d0 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.328+0000 7fcf6dffb640 1 -- 172.21.15.31:0/97838164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68014a00 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.329+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf700c5430 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.330+0000 7fcf6dffb640 1 -- 172.21.15.31:0/97838164 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fcf68014b60 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.331+0000 7fcf6dffb640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf5c06aed0 0x7fcf5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.331+0000 7fcf6dffb640 1 -- 172.21.15.31:0/97838164 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf6808a640 con 0x7fcf700cf570 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.331+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf5c06aed0 0x7fcf5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.331+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf5c06aed0 0x7fcf5c06d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf701e7050 tx=0x7fcf64009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.331+0000 7fcf7a374740 1 Processor -- start 2024-01-19T04:55:36.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7a374740 1 -- start start 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe369a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe369a770 0x55bfe36ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7a374740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe365b010 con 0x55bfe369a770 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe360ce60 con 0x55bfe36c7290 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfe360a550 con 0x55bfe36c7670 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf75d4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe369a770 0x55bfe36ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf75d4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe369a770 0x55bfe36ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43424/0 (socket says 172.21.15.31:43424) 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1494762812 learned_addr learned my addr 172.21.15.31:0/1494762812 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf7554c640 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe369a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.332+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 msgr2=0x55bfe36ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 msgr2=0x55bfe369a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe369a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf75d4d640 1 -- 172.21.15.31:0/1494762812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfe35fdb00 con 0x55bfe369a770 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7554c640 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe369a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe369a770 0x55bfe36ca560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf600090b0 tx=0x7fcf60008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbc609ee9e6afa8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf56ffd640 1 -- 172.21.15.31:0/1494762812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6003e050 con 0x55bfe369a770 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf56ffd640 1 -- 172.21.15.31:0/1494762812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6003f040 con 0x55bfe369a770 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf56ffd640 1 -- 172.21.15.31:0/1494762812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60039450 con 0x55bfe369a770 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7a374740 1 -- 172.21.15.31:0/1494762812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe369a770 msgr2=0x55bfe36ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7a374740 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe369a770 0x55bfe36ca560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf600090b0 tx=0x7fcf60008030 comp rx=0 tx=0).stop 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7a374740 1 -- 172.21.15.31:0/1494762812 shutdown_connections 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7a374740 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe369a770 0x55bfe36ca560 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7a374740 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7a374740 1 --2- 172.21.15.31:0/1494762812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe369a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.333+0000 7fcf7a374740 1 -- 172.21.15.31:0/1494762812 >> 172.21.15.31:0/1494762812 conn(0x55bfe369c4a0 msgr2=0x55bfe36e5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.334+0000 7fcf7a374740 1 -- 172.21.15.31:0/1494762812 shutdown_connections 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.334+0000 7fcf7a374740 1 -- 172.21.15.31:0/1494762812 wait complete. 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.334+0000 7fcf7a374740 1 -- start start 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.334+0000 7fcf7a374740 1 Processor -- start 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe36c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36bf2a0 0x55bfe36cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7a374740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe3605e40 con 0x55bfe36bf2a0 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe35bf250 con 0x55bfe36c7290 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfe35beef0 con 0x55bfe36c7670 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7a374740 10 client.0 _ll_register_callbacks cb 0x55bfe35c3720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36bf2a0 0x55bfe36cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36bf2a0 0x55bfe36cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43426/0 (socket says 172.21.15.31:43426) 2024-01-19T04:55:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 -- 172.21.15.31:0/3410073841 learned_addr learned my addr 172.21.15.31:0/3410073841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf7554c640 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe36c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 -- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 msgr2=0x55bfe36cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 -- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 msgr2=0x55bfe36c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe36c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.335+0000 7fcf75d4d640 1 -- 172.21.15.31:0/3410073841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf700d0290 con 0x55bfe36bf2a0 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36bf2a0 0x55bfe36cdbe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009990 tx=0x7fcf60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c950b6857ebe7214 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf4ffff640 1 -- 172.21.15.31:0/3410073841 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60003610 con 0x55bfe36bf2a0 2024-01-19T04:55:36.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf4ffff640 1 -- 172.21.15.31:0/3410073841 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf60003770 con 0x55bfe36bf2a0 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfe35fecd0 con 0x55bfe36bf2a0 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf4ffff640 1 -- 172.21.15.31:0/3410073841 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6002c080 con 0x55bfe36bf2a0 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 10 client.4701 fetch_fsmap learned FSMap version 6 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55bfe36044c0 con 0x55bfe36bf2a0 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf4ffff640 1 -- 172.21.15.31:0/3410073841 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bfe35fecd0 con 0x55bfe36bf2a0 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf4ffff640 1 -- 172.21.15.31:0/3410073841 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf60003930 con 0x55bfe36bf2a0 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 10 client.4701 fetch_fsmap finished waiting for FSMap version 6 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 10 client.4701 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 20 client.4701 populate_metadata read hostname 'smithi031' 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe37001e0 0x55bfe37025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 4 client.4701 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 --> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfe35fe9f0 con 0x55bfe37001e0 2024-01-19T04:55:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.336+0000 7fcf7554c640 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe37001e0 0x55bfe37025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf4ffff640 10 client.4701 ms_handle_connect on v2:172.21.15.31:6832/2576238027 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7554c640 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe37001e0 0x55bfe37025d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=338952762ae573a server_cookie=1d97f3854f8a1519 in_seq=0 out_seq=0 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf4ffff640 1 -- 172.21.15.31:0/3410073841 <== mds.0 v2:172.21.15.31:6832/2576238027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfe35fe9f0 con 0x55bfe37001e0 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf4ffff640 10 client.4701 handle_command_reply: tid=0 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 client.4701 shutdown 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36bf2a0 msgr2=0x55bfe36cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36bf2a0 0x55bfe36cdbe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009990 tx=0x7fcf60004730 comp rx=0 tx=0).stop 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 shutdown_connections 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe37001e0 0x55bfe37025d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36bf2a0 0x55bfe36cdbe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf4ffff640 0 client.4701 ms_handle_reset on v2:172.21.15.31:6832/2576238027 2024-01-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 --2- 172.21.15.31:0/3410073841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36c7290 0x55bfe36c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 >> 172.21.15.31:0/3410073841 conn(0x55bfe369a770 msgr2=0x55bfe36e8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 shutdown_connections 2024-01-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.337+0000 7fcf7a374740 1 -- 172.21.15.31:0/3410073841 wait complete. 2024-01-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.338+0000 7fcf7a374740 20 client.4701 trim_cache size 0 max 16384 2024-01-19T04:55:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.353+0000 7fcf7a374740 1 Processor -- start 2024-01-19T04:55:36.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.353+0000 7fcf7a374740 1 -- start start 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe369df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe369e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe369fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf7a374740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe369ae40 con 0x55bfe36c7290 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe3584300 con 0x55bfe36bf2a0 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfe36997d0 con 0x55bfe36c7670 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe369e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe369e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43438/0 (socket says 172.21.15.31:43438) 2024-01-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 -- 172.21.15.31:0/7687299 learned_addr learned my addr 172.21.15.31:0/7687299 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf7554c640 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe369df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 -- 172.21.15.31:0/7687299 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 msgr2=0x55bfe369fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe369fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 -- 172.21.15.31:0/7687299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 msgr2=0x55bfe369df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe369df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 -- 172.21.15.31:0/7687299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfe35a1150 con 0x55bfe36c7290 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.354+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe369e470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf700b0000 tx=0x7fcf64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8a1ac83fa11ae8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf567fc640 1 -- 172.21.15.31:0/7687299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64006ba0 con 0x55bfe36c7290 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf567fc640 1 -- 172.21.15.31:0/7687299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64006d00 con 0x55bfe36c7290 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf567fc640 1 -- 172.21.15.31:0/7687299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64007230 con 0x55bfe36c7290 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 -- 172.21.15.31:0/7687299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 msgr2=0x55bfe369e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe369e470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf700b0000 tx=0x7fcf64012000 comp rx=0 tx=0).stop 2024-01-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 -- 172.21.15.31:0/7687299 shutdown_connections 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe369fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe369e470 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 --2- 172.21.15.31:0/7687299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe369df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 -- 172.21.15.31:0/7687299 >> 172.21.15.31:0/7687299 conn(0x55bfe369a770 msgr2=0x55bfe36959a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 -- 172.21.15.31:0/7687299 shutdown_connections 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.355+0000 7fcf7a374740 1 -- 172.21.15.31:0/7687299 wait complete. 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.356+0000 7fcf7a374740 1 -- start start 2024-01-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.356+0000 7fcf7a374740 1 Processor -- start 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe36e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe374ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe36cd0a0 con 0x55bfe36c7290 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfe374f210 con 0x55bfe36bf2a0 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfe36e5290 con 0x55bfe36c7670 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 10 client.0 _ll_register_callbacks cb 0x7fcf60006a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe374ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe374ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43454/0 (socket says 172.21.15.31:43454) 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 -- 172.21.15.31:0/832268336 learned_addr learned my addr 172.21.15.31:0/832268336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7554c640 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe36e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf75d4d640 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 -- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 msgr2=0x55bfe36cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 -- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 msgr2=0x55bfe36e4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe36e4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 -- 172.21.15.31:0/832268336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf700cf9f0 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf74d4b640 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe374ecd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcf700b0410 tx=0x7fcf64000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1f13a50609d054 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf4ffff640 1 -- 172.21.15.31:0/832268336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64006ba0 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf4ffff640 1 -- 172.21.15.31:0/832268336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf6401b030 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf6003f040 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf4ffff640 1 -- 172.21.15.31:0/832268336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf64006e70 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 10 client.4707 fetch_fsmap learned FSMap version 6 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.357+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55bfe35c6940 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf4ffff640 1 -- 172.21.15.31:0/832268336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6003f040 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf4ffff640 1 -- 172.21.15.31:0/832268336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf64012070 con 0x55bfe36c7290 2024-01-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 10 client.4707 fetch_fsmap finished waiting for FSMap version 6 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 10 client.4707 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 20 client.4707 populate_metadata read hostname 'smithi031' 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe36c22c0 0x55bfe36c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 4 client.4707 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 --> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bfe33056c0 con 0x55bfe36c22c0 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7554c640 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe36c22c0 0x55bfe36c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf4ffff640 10 client.4707 ms_handle_connect on v2:172.21.15.31:6832/2576238027 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7554c640 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe36c22c0 0x55bfe36c26a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d9b6a0cf653fb5 server_cookie=6aa8ac00a4c7af4b in_seq=0 out_seq=0 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf4ffff640 1 -- 172.21.15.31:0/832268336 <== mds.0 v2:172.21.15.31:6832/2576238027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55bfe33056c0 con 0x55bfe36c22c0 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf4ffff640 10 client.4707 handle_command_reply: tid=0 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 client.4707 shutdown 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 msgr2=0x55bfe374ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe374ecd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcf700b0410 tx=0x7fcf64000ed0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 shutdown_connections 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:6832/2576238027,v1:172.21.15.31:6833/2576238027] conn(0x55bfe36c22c0 0x55bfe36c26a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bfe36c7670 0x55bfe36cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe36c7290 0x55bfe374ecd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf4ffff640 0 client.4707 ms_handle_reset on v2:172.21.15.31:6832/2576238027 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 --2- 172.21.15.31:0/832268336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe36bf2a0 0x55bfe36e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.358+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 >> 172.21.15.31:0/832268336 conn(0x55bfe369a770 msgr2=0x55bfe3699ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 shutdown_connections 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf7a374740 1 -- 172.21.15.31:0/832268336 wait complete. 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf7a374740 20 client.4707 trim_cache size 0 max 16384 2024-01-19T04:55:36.364 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "16b8b5c3-25df-408f-9c77-cd32e346926d", 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4290, 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 16.84554996, 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 6, 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 20, 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 20, 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 23.874769134000001 2024-01-19T04:55:36.365 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:55:36.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf5c06aed0 msgr2=0x7fcf5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf5c06aed0 0x7fcf5c06d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf701e7050 tx=0x7fcf64009040 comp rx=0 tx=0).stop 2024-01-19T04:55:36.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf700cf570 msgr2=0x7fcf700b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf700cf570 0x7fcf700b5c20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800efc0 tx=0x7fcf6800c550 comp rx=0 tx=0).stop 2024-01-19T04:55:36.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 shutdown_connections 2024-01-19T04:55:36.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf5c06aed0 0x7fcf5c06d390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf70161860 0x7fcf700c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf7015d3a0 0x7fcf701e7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 --2- 172.21.15.31:0/97838164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf700cf570 0x7fcf700b5c20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 >> 172.21.15.31:0/97838164 conn(0x7fcf700d1100 msgr2=0x7fcf700add30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 shutdown_connections 2024-01-19T04:55:36.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.359+0000 7fcf777b0640 1 -- 172.21.15.31:0/97838164 wait complete. 2024-01-19T04:55:36.369 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-01-19T04:55:36.393 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-01-19T04:55:36.393 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-19T04:55:36.393 INFO:tasks.ceph_fuse:client.0 config is {} 2024-01-19T04:55:36.393 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-19T04:55:36.393 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:36.393 INFO:tasks.ceph_fuse:client.2 config is {} 2024-01-19T04:55:36.393 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-19T04:55:36.393 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-19T04:55:36.393 INFO:tasks.ceph_fuse:client.1 config is {} 2024-01-19T04:55:36.393 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-19T04:55:36.394 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:36.394 INFO:tasks.ceph_fuse:client.3 config is {} 2024-01-19T04:55:36.394 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-19T04:55:36.394 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-19T04:55:36.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:36.394 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-01-19T04:55:36.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:36.413 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link delete ceph-brx 2024-01-19T04:55:36.487 INFO:teuthology.orchestra.run.smithi137.stderr:Cannot find device "ceph-brx" 2024-01-19T04:55:36.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:55:36.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:36.489 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:55:36.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:36.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-01-19T04:55:36.574 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-01-19T04:55:36.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:55:36.575 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-01-19T04:55:36.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:55:36.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.717+0000 7f9c8dbec640 1 Processor -- start 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.717+0000 7f9c8dbec640 1 -- start start 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c8dbec640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9c88157b10 0x7f9c8815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c8dbec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 0x7f9c880cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c8dbec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c881611e0 con 0x7f9c880cec90 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c8dbec640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c881614c0 con 0x7f9c88157b10 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c8dbec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c881617a0 con 0x7f9c88070740 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c86ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9c880cec90 0x7f9c88070030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59304/0 (socket says 172.21.15.31:59304) 2024-01-19T04:55:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c86ffd640 1 -- 172.21.15.31:0/597450903 learned_addr learned my addr 172.21.15.31:0/597450903 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c87fff640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 0x7f9c880cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.718+0000 7f9c877fe640 1 --2- 172.21.15.31:0/597450903 >> v2:172.21.15.137:3300/0 conn(0x7f9c88157b10 0x7f9c8815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122700640 0 0) 0x7f9c881611e0 con 0x7f9c880cec90 2024-01-19T04:55:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c70003520 con 0x7f9c880cec90 2024-01-19T04:55:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 135642650 0 0) 0x7f9c70003520 con 0x7f9c880cec90 2024-01-19T04:55:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 msgr2=0x7f9c880cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 0x7f9c880cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 >> v2:172.21.15.137:3300/0 conn(0x7f9c88157b10 msgr2=0x7f9c8815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 --2- 172.21.15.31:0/597450903 >> v2:172.21.15.137:3300/0 conn(0x7f9c88157b10 0x7f9c8815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88161a80 con 0x7f9c880cec90 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f9c7c001670 con 0x7f9c880cec90 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 >> v1:172.21.15.31:6789/0 conn(0x7f9c880cec90 legacy=0x7f9c88070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c70003970 0x7f9c70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c700047c0 0x7f9c70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 0x7f9c70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c881614c0 con 0x7f9c700047c0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c881617a0 con 0x7f9c70003970 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7000b980 con 0x7f9c88070740 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c86ffd640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c700047c0 0x7f9c70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c877fe640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c70003970 0x7f9c70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c87fff640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 0x7f9c70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c86ffd640 1 -- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 msgr2=0x7f9c70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c86ffd640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c88070740 0x7f9c70005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c86ffd640 1 -- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c70003970 msgr2=0x7f9c70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c86ffd640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c70003970 0x7f9c70003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.719+0000 7f9c86ffd640 1 -- 172.21.15.31:0/597450903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70003520 con 0x7f9c700047c0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.720+0000 7f9c86ffd640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c700047c0 0x7f9c70004c40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c290 tx=0x7f9c700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5bcab71546fce9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.720+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c005ad0 con 0x7f9c700047c0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.720+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c005c30 con 0x7f9c700047c0 2024-01-19T04:55:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.720+0000 7f9c867fc640 1 -- 172.21.15.31:0/597450903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c02c8d0 con 0x7f9c700047c0 2024-01-19T04:55:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.720+0000 7f9c8dbec640 1 -- 172.21.15.31:0/597450903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c700047c0 msgr2=0x7f9c70004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.720+0000 7f9c8dbec640 1 --2- 172.21.15.31:0/597450903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c700047c0 0x7f9c70004c40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c290 tx=0x7f9c700057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.720+0000 7f9c87fff640 1 -- 172.21.15.31:0/597450903 reap_dead start 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.721+0000 7f9c8dbec640 1 -- 172.21.15.31:0/597450903 shutdown_connections 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.721+0000 7f9c8dbec640 1 -- 172.21.15.31:0/597450903 >> 172.21.15.31:0/597450903 conn(0x7f9c880d10c0 msgr2=0x7f9c880d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.721+0000 7f9c8dbec640 1 -- 172.21.15.31:0/597450903 shutdown_connections 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.721+0000 7f9c8dbec640 1 -- 172.21.15.31:0/597450903 wait complete. 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.722+0000 7f9c8dbec640 1 Processor -- start 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.722+0000 7f9c8dbec640 1 -- start start 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c8dbec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c700047c0 0x7f9c881f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c8dbec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c88153830 0x7f9c880c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c8dbec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880c86d0 0x7f9c880c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c8dbec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b5830 con 0x7f9c88153830 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c8dbec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b53e0 con 0x7f9c700047c0 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c8dbec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880c8d00 con 0x7f9c880c86d0 2024-01-19T04:55:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c88153830 0x7f9c880c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c88153830 0x7f9c880c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43482/0 (socket says 172.21.15.31:43482) 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 -- 172.21.15.31:0/3319618670 learned_addr learned my addr 172.21.15.31:0/3319618670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c877fe640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c700047c0 0x7f9c881f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c87fff640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880c86d0 0x7f9c880c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 -- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880c86d0 msgr2=0x7f9c880c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880c86d0 0x7f9c880c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 -- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c700047c0 msgr2=0x7f9c881f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c700047c0 0x7f9c881f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 -- 172.21.15.31:0/3319618670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880c8fe0 con 0x7f9c88153830 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c86ffd640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c88153830 0x7f9c880c8190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c001930 tx=0x7f9c7c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c6ffff640 1 -- 172.21.15.31:0/3319618670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c039070 con 0x7f9c88153830 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c6ffff640 1 -- 172.21.15.31:0/3319618670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c005e90 con 0x7f9c88153830 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c880c92c0 con 0x7f9c88153830 2024-01-19T04:55:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.723+0000 7f9c6ffff640 1 -- 172.21.15.31:0/3319618670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c02dea0 con 0x7f9c88153830 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.724+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c88153cb0 con 0x7f9c88153830 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.724+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c88074a80 con 0x7f9c88153830 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.725+0000 7f9c6ffff640 1 -- 172.21.15.31:0/3319618670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9c7c032070 con 0x7f9c88153830 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.725+0000 7f9c6ffff640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c7406aed0 0x7f9c7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.725+0000 7f9c6ffff640 1 -- 172.21.15.31:0/3319618670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c7c03c040 con 0x7f9c88153830 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.725+0000 7f9c877fe640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c7406aed0 0x7f9c7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.726+0000 7f9c877fe640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c7406aed0 0x7f9c7406d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009bc0 tx=0x7f9c78009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.729+0000 7f9c6ffff640 1 -- 172.21.15.31:0/3319618670 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9c7c0759f0 con 0x7f9c88153830 2024-01-19T04:55:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.944+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c880c8ab0 con 0x7f9c88153830 2024-01-19T04:55:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.945+0000 7f9c6ffff640 1 -- 172.21.15.31:0/3319618670 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9c7c045070 con 0x7f9c88153830 2024-01-19T04:55:36.944 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:55:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c7406aed0 msgr2=0x7f9c7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c7406aed0 0x7f9c7406d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009bc0 tx=0x7f9c78009190 comp rx=0 tx=0).stop 2024-01-19T04:55:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c88153830 msgr2=0x7f9c880c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c88153830 0x7f9c880c8190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c001930 tx=0x7f9c7c000fe0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 shutdown_connections 2024-01-19T04:55:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c7406aed0 0x7f9c7406d390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880c86d0 0x7f9c880c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c88153830 0x7f9c880c8190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 --2- 172.21.15.31:0/3319618670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c700047c0 0x7f9c881f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 >> 172.21.15.31:0/3319618670 conn(0x7f9c880d10c0 msgr2=0x7f9c880b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 shutdown_connections 2024-01-19T04:55:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:36.946+0000 7f9c8dbec640 1 -- 172.21.15.31:0/3319618670 wait complete. 2024-01-19T04:55:36.956 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:55:36.957 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:55:36.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:36.957 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42756sec preferred_lft 42756sec 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:55:36.974 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-19T04:55:36.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:36.974 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:36.974 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:36.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-01-19T04:55:36.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-01-19T04:55:36.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-01-19T04:55:36.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-19T04:55:36.974 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:37.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.137 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:55:37.163 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:55:37.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.165 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:55:37.251 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:55:37.251 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:55:37.251 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:55:37.251 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:55:37.251 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:55:37.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.252 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:37.252 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:37.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:55:37.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:55:37.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:55:37.252 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:37.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.472 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:55:37.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.487 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-01-19T04:55:37.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.543 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:37.543 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:37.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:37.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-19T04:55:37.543 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:37.628 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-19T04:55:37.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.628 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:37.631 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:37.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-19T04:55:37.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-19T04:55:37.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-19T04:55:37.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-19T04:55:37.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-19T04:55:37.631 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:37.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:37.769 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:37.769 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:37.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-01-19T04:55:37.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-01-19T04:55:37.770 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:37.811 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-19T04:55:37.811 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:37.811 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:37.866 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:55:37.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:37.867 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:37.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:55:37.989 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:55:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:55:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:55:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:55:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:38.046 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:38.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:38.047 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:55:38.098 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:55:38.098 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-19T04:55:38.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:55:38.167 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:38.222 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:55:38.222 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:55:38.222 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:55:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:38.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:38.225 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:55:38.232 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-01-19T04:55:38.232+0000 7f7e62438480 -1 init, newargv = 0x55a50f753660 newargc=15 2024-01-19T04:55:38.232 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[45304]: starting ceph client 2024-01-19T04:55:38.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[45304]: starting fuse 2024-01-19T04:55:38.247 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:55:38.247 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:55:38.247 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:55:38.248 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:55:38.249 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:55:38.249 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:38.249 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-01-19T04:55:38.249 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:55:38.367 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.45304.asok 2024-01-19T04:55:38.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:55:38.378 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45304.asok 2024-01-19T04:55:38.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:38.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45304.asok status 2024-01-19T04:55:38.515 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "45304", 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:55:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4719, 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 4719 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2106660531 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:55:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2106660531 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.4719 192.168.0.1:0/2106660531", 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2106660531", 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 6, 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 20, 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:55:38.518 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:55:38.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:55:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.625+0000 7f22c06fc640 1 Processor -- start 2024-01-19T04:55:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.625+0000 7f22c06fc640 1 -- start start 2024-01-19T04:55:38.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22c06fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22b80ce760 0x7f22b81449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:38.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22c06fc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f22b80ceb40 0x7f22b80cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:38.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22c06fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b80cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:38.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22c06fc640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b815f3e0 con 0x7f22b80ce760 2024-01-19T04:55:38.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22c06fc640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b815f6c0 con 0x7f22b80ceb40 2024-01-19T04:55:38.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22c06fc640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b815f9a0 con 0x7f22b815ee40 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22be498640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22b80ce760 0x7f22b81449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22be498640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22b80ce760 0x7f22b81449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43486/0 (socket says 172.21.15.31:43486) 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22be498640 1 -- 172.21.15.31:0/2045619866 learned_addr learned my addr 172.21.15.31:0/2045619866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22bec99640 1 --2- 172.21.15.31:0/2045619866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b80cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.626+0000 7f22bdc97640 1 --2- 172.21.15.31:0/2045619866 >> v2:172.21.15.137:3300/0 conn(0x7f22b80ceb40 0x7f22b80cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22be498640 1 -- 172.21.15.31:0/2045619866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 msgr2=0x7f22b80cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22be498640 1 --2- 172.21.15.31:0/2045619866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b80cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22be498640 1 -- 172.21.15.31:0/2045619866 >> v2:172.21.15.137:3300/0 conn(0x7f22b80ceb40 msgr2=0x7f22b80cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22be498640 1 --2- 172.21.15.31:0/2045619866 >> v2:172.21.15.137:3300/0 conn(0x7f22b80ceb40 0x7f22b80cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22be498640 1 -- 172.21.15.31:0/2045619866 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b815fc80 con 0x7f22b80ce760 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22be498640 1 --2- 172.21.15.31:0/2045619866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b80ce760 0x7f22b81449d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f22b80cc430 tx=0x7f22b002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4f9dcfe2c8e20b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22bd496640 1 -- 172.21.15.31:0/2045619866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b002ca70 con 0x7f22b80ce760 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.627+0000 7f22bd496640 1 -- 172.21.15.31:0/2045619866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b002cbd0 con 0x7f22b80ce760 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22bd496640 1 -- 172.21.15.31:0/2045619866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00343f0 con 0x7f22b80ce760 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 -- 172.21.15.31:0/2045619866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b80ce760 msgr2=0x7f22b81449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 --2- 172.21.15.31:0/2045619866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b80ce760 0x7f22b81449d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f22b80cc430 tx=0x7f22b002bed0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 -- 172.21.15.31:0/2045619866 shutdown_connections 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 --2- 172.21.15.31:0/2045619866 >> v2:172.21.15.137:3300/0 conn(0x7f22b80ceb40 0x7f22b80cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 --2- 172.21.15.31:0/2045619866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b80cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 --2- 172.21.15.31:0/2045619866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b80ce760 0x7f22b81449d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 -- 172.21.15.31:0/2045619866 >> 172.21.15.31:0/2045619866 conn(0x7f22b806a790 msgr2=0x7f22b806aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 -- 172.21.15.31:0/2045619866 shutdown_connections 2024-01-19T04:55:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.628+0000 7f22c06fc640 1 -- 172.21.15.31:0/2045619866 wait complete. 2024-01-19T04:55:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.629+0000 7f22c06fc640 1 Processor -- start 2024-01-19T04:55:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.629+0000 7f22c06fc640 1 -- start start 2024-01-19T04:55:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22c06fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b80ce760 0x7f22b8070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22c06fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b81394d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22c06fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8153040 0x7f22b8139a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22c06fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80c8c70 con 0x7f22b8153040 2024-01-19T04:55:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22c06fc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80c88d0 con 0x7f22b80ce760 2024-01-19T04:55:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22c06fc640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8071660 con 0x7f22b815ee40 2024-01-19T04:55:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bec99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8153040 0x7f22b8139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bec99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8153040 0x7f22b8139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43502/0 (socket says 172.21.15.31:43502) 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bec99640 1 -- 172.21.15.31:0/770981398 learned_addr learned my addr 172.21.15.31:0/770981398 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22be498640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b80ce760 0x7f22b8070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bdc97640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b81394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bec99640 1 -- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 msgr2=0x7f22b81394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bec99640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b81394d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bec99640 1 -- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b80ce760 msgr2=0x7f22b8070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.630+0000 7f22bec99640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b80ce760 0x7f22b8070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.631+0000 7f22bec99640 1 -- 172.21.15.31:0/770981398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0009b30 con 0x7f22b8153040 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.631+0000 7f22bec99640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8153040 0x7f22b8139a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22b400d5d0 tx=0x7f22b400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.631+0000 7f22aeffd640 1 -- 172.21.15.31:0/770981398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40125f0 con 0x7f22b8153040 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.631+0000 7f22aeffd640 1 -- 172.21.15.31:0/770981398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b4012770 con 0x7f22b8153040 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.631+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b8071940 con 0x7f22b8153040 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.631+0000 7f22aeffd640 1 -- 172.21.15.31:0/770981398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4012b20 con 0x7f22b8153040 2024-01-19T04:55:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.631+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b8071350 con 0x7f22b8153040 2024-01-19T04:55:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.632+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b80c8db0 con 0x7f22b8153040 2024-01-19T04:55:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.632+0000 7f22aeffd640 1 -- 172.21.15.31:0/770981398 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f22b4012c80 con 0x7f22b8153040 2024-01-19T04:55:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.633+0000 7f22aeffd640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f229c06aed0 0x7f229c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.633+0000 7f22aeffd640 1 -- 172.21.15.31:0/770981398 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22b4089920 con 0x7f22b8153040 2024-01-19T04:55:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.633+0000 7f22be498640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f229c06aed0 0x7f229c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.633+0000 7f22be498640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f229c06aed0 0x7f229c06d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f22b0035000 tx=0x7f22b0030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:38.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.637+0000 7f22aeffd640 1 -- 172.21.15.31:0/770981398 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f22b80c8db0 con 0x7f22b8153040 2024-01-19T04:55:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.852+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22b80c8db0 con 0x7f22b8153040 2024-01-19T04:55:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.852+0000 7f22aeffd640 1 -- 172.21.15.31:0/770981398 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f22b80c8db0 con 0x7f22b8153040 2024-01-19T04:55:38.852 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f229c06aed0 msgr2=0x7f229c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f229c06aed0 0x7f229c06d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f22b0035000 tx=0x7f22b0030000 comp rx=0 tx=0).stop 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8153040 msgr2=0x7f22b8139a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8153040 0x7f22b8139a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22b400d5d0 tx=0x7f22b400da90 comp rx=0 tx=0).stop 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 shutdown_connections 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f229c06aed0 0x7f229c06d390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8153040 0x7f22b8139a10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22b815ee40 0x7f22b81394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 --2- 172.21.15.31:0/770981398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b80ce760 0x7f22b8070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:38.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 >> 172.21.15.31:0/770981398 conn(0x7f22b806a790 msgr2=0x7f22b80cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:38.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 shutdown_connections 2024-01-19T04:55:38.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:38.854+0000 7f22c06fc640 1 -- 172.21.15.31:0/770981398 wait complete. 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:55:38.864 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:55:38.865 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-19T04:55:38.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:38.865 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:38.880 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42754sec preferred_lft 42754sec 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:38.881 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c57:fcff:feba:867a/64 scope link tentative 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::98a9:56ff:fef7:e409/64 scope link tentative 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:38.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:38.882 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:55:38.946 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:55:38.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:38.947 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:55:39.003 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:55:39.003 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:55:39.003 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:55:39.003 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:55:39.003 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:55:39.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.003 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:39.003 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:39.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:55:39.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:55:39.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:55:39.004 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:39.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.098 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:55:39.113 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:55:39.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.113 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-01-19T04:55:39.168 INFO:teuthology.orchestra.run.smithi031.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-19T04:55:39.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.168 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:39.168 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:39.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:55:39.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-19T04:55:39.169 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:39.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-19T04:55:39.316 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.317 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:55:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ea:ef:a2:b5:31:99 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-19T04:55:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-19T04:55:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::e8ef:a2ff:feb5:3199/64 scope link 2024-01-19T04:55:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-19T04:55:39.384 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:39.384 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:39.384 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:39.384 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ea:ef:a2:b5:31:99 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::e8ef:a2ff:feb5:3199/64 scope link 2024-01-19T04:55:39.385 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:39.386 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-19T04:55:39.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.386 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:39.386 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:39.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-19T04:55:39.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-19T04:55:39.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-19T04:55:39.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-19T04:55:39.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-19T04:55:39.387 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:39.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.511 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:55:39.511 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:55:39.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.1 up 2024-01-19T04:55:39.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.1 master ceph-brx 2024-01-19T04:55:39.511 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:55:39.552 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-01-19T04:55:39.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:39.552 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-19T04:55:39.606 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-19T04:55:39.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:39.607 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-19T04:55:39.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:55:39.728 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:55:39.784 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:55:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:55:39.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.786 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:55:39.839 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:55:39.839 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-01-19T04:55:39.839 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-01-19T04:55:39.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:55:39.907 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:39.963 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:55:39.963 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:55:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:55:39.966 INFO:teuthology.orchestra.run.smithi031.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:39.966 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:55:39.966 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:55:39.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:39.966 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:55:39.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi031.stderr:2024-01-19T04:55:39.967+0000 7f2def7d9480 -1 init, newargv = 0x5625080c2540 newargc=15 2024-01-19T04:55:39.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi031.stderr:ceph-fuse[45798]: starting ceph client 2024-01-19T04:55:39.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi031.stderr:ceph-fuse[45798]: starting fuse 2024-01-19T04:55:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:55:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:44 2024-01-19T04:55:39.981 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-01-19T04:55:39.981 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:55:39.981 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:55:39.981 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:55:39.981 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:55:39.982 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:55:39.983 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:55:39.983 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:55:39.983 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:55:39.983 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.2")) 2024-01-19T04:55:39.983 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:55:40.100 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.2.45798.asok 2024-01-19T04:55:40.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:55:40.110 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45798.asok 2024-01-19T04:55:40.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:40.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45798.asok status 2024-01-19T04:55:40.246 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "2", 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "45798", 2024-01-19T04:55:40.247 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:55:40.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:40.248 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:55:40.248 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:55:40.248 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4734, 2024-01-19T04:55:40.248 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:55:40.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:55:40.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 4734 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.2:0", 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2190920584 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.2:0", 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2190920584 2024-01-19T04:55:40.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.4734 192.168.0.2:0/2190920584", 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.2:0/2190920584", 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 6, 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 20, 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:55:40.250 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:55:40.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:55:40.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.356+0000 7fc560ad0640 1 Processor -- start 2024-01-19T04:55:40.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.356+0000 7fc560ad0640 1 -- start start 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.356+0000 7fc560ad0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 0x7fc55c143930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.356+0000 7fc560ad0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c142890 con 0x7fc55c142320 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.356+0000 7fc560ad0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c142b70 con 0x7fc55c0cc230 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.356+0000 7fc560ad0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c142e50 con 0x7fc55c0b3450 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc55ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc55c142320 0x7fc55c143e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53700/0 (socket says 172.21.15.31:53700) 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc55ad9d640 1 -- 172.21.15.31:0/834515336 learned_addr learned my addr 172.21.15.31:0/834515336 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc551d9b640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 0x7fc55c143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684744026 0 0) 0x7fc55c142890 con 0x7fc55c142320 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc548003520 con 0x7fc55c142320 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 198738021 0 0) 0x7fc548003520 con 0x7fc55c142320 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 msgr2=0x7fc55c143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 0x7fc55c143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 >> v1:172.21.15.137:6789/0 conn(0x7fc55c0cc230 legacy=0x7fc55c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c143130 con 0x7fc55c142320 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fc5540019f0 con 0x7fc55c142320 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 >> v1:172.21.15.31:6789/0 conn(0x7fc55c142320 legacy=0x7fc55c143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480039a0 0x7fc548003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5480047c0 0x7fc548004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 0x7fc548005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c142e50 con 0x7fc5480039a0 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54800ba30 con 0x7fc5480047c0 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54800bbf0 con 0x7fc55c0b3450 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc55a59c640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480039a0 0x7fc548003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc551d9b640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5480047c0 0x7fc548004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.357+0000 7fc55ad9d640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 0x7fc548005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc55a59c640 1 -- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 msgr2=0x7fc548005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc55a59c640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 0x7fc548005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc55a59c640 1 -- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5480047c0 msgr2=0x7fc548004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc55a59c640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5480047c0 0x7fc548004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc55a59c640 1 -- 172.21.15.31:0/834515336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc548003520 con 0x7fc5480039a0 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc55ad9d640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc55c0b3450 0x7fc548005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc55a59c640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480039a0 0x7fc548003dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc54c001720 tx=0x7fc5480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7c50ba59402ae0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0052d0 con 0x7fc5480039a0 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c02e050 con 0x7fc5480039a0 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.358+0000 7fc559d9b640 1 -- 172.21.15.31:0/834515336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0035f0 con 0x7fc5480039a0 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.359+0000 7fc560ad0640 1 -- 172.21.15.31:0/834515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480039a0 msgr2=0x7fc548003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.359+0000 7fc560ad0640 1 --2- 172.21.15.31:0/834515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480039a0 0x7fc548003dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc54c001720 tx=0x7fc5480057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.359+0000 7fc55ad9d640 1 -- 172.21.15.31:0/834515336 reap_dead start 2024-01-19T04:55:40.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.359+0000 7fc560ad0640 1 -- 172.21.15.31:0/834515336 shutdown_connections 2024-01-19T04:55:40.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.359+0000 7fc560ad0640 1 -- 172.21.15.31:0/834515336 >> 172.21.15.31:0/834515336 conn(0x7fc55c06abe0 msgr2=0x7fc55c066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:40.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.359+0000 7fc560ad0640 1 -- 172.21.15.31:0/834515336 shutdown_connections 2024-01-19T04:55:40.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.359+0000 7fc560ad0640 1 -- 172.21.15.31:0/834515336 wait complete. 2024-01-19T04:55:40.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.360+0000 7fc560ad0640 1 Processor -- start 2024-01-19T04:55:40.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.360+0000 7fc560ad0640 1 -- start start 2024-01-19T04:55:40.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc560ad0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5480039a0 0x7fc55c15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc560ad0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0c8330 0x7fc55c0c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc560ad0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc55c15a1b0 0x7fc55c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc560ad0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c15e600 con 0x7fc55c15a1b0 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc560ad0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c8cd0 con 0x7fc55c0c8330 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc560ad0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c15aba0 con 0x7fc5480039a0 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc55ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc55c15a1b0 0x7fc55c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc55ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc55c15a1b0 0x7fc55c15a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53792/0 (socket says 172.21.15.31:53792) 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc55ad9d640 1 -- 172.21.15.31:0/3343524247 learned_addr learned my addr 172.21.15.31:0/3343524247 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc551d9b640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0c8330 0x7fc55c0c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc55a59c640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5480039a0 0x7fc55c15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.361+0000 7fc55ad9d640 1 -- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5480039a0 msgr2=0x7fc55c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc55ad9d640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5480039a0 0x7fc55c15d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc55ad9d640 1 -- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0c8330 msgr2=0x7fc55c0c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc55ad9d640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0c8330 0x7fc55c0c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc55ad9d640 1 -- 172.21.15.31:0/3343524247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c0c8f10 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc551d9b640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0c8330 0x7fc55c0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc55ad9d640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc55c15a1b0 0x7fc55c15a660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc554002010 tx=0x7fc55400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc5537fe640 1 -- 172.21.15.31:0/3343524247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554007ae0 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc5537fe640 1 -- 172.21.15.31:0/3343524247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55400ed40 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c0c91f0 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc5537fe640 1 -- 172.21.15.31:0/3343524247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554013620 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.362+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c0c97b0 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.363+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c0696e0 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.363+0000 7fc5537fe640 1 -- 172.21.15.31:0/3343524247 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc554013780 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.364+0000 7fc5537fe640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc53c06aed0 0x7fc53c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.364+0000 7fc5537fe640 1 -- 172.21.15.31:0/3343524247 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc554089bc0 con 0x7fc55c15a1b0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.364+0000 7fc55a59c640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc53c06aed0 0x7fc53c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:40.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.365+0000 7fc55a59c640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc53c06aed0 0x7fc53c06d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc54c033000 tx=0x7fc54c009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:40.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.368+0000 7fc5537fe640 1 -- 172.21.15.31:0/3343524247 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc554054960 con 0x7fc55c15a1b0 2024-01-19T04:55:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.585+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc55c0c7a50 con 0x7fc55c15a1b0 2024-01-19T04:55:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.585+0000 7fc5537fe640 1 -- 172.21.15.31:0/3343524247 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc554057f80 con 0x7fc55c15a1b0 2024-01-19T04:55:40.585 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:55:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc53c06aed0 msgr2=0x7fc53c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc53c06aed0 0x7fc53c06d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc54c033000 tx=0x7fc54c009030 comp rx=0 tx=0).stop 2024-01-19T04:55:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc55c15a1b0 msgr2=0x7fc55c15a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc55c15a1b0 0x7fc55c15a660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc554002010 tx=0x7fc55400e530 comp rx=0 tx=0).stop 2024-01-19T04:55:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 shutdown_connections 2024-01-19T04:55:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc53c06aed0 0x7fc53c06d390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc55c15a1b0 0x7fc55c15a660 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0c8330 0x7fc55c0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 --2- 172.21.15.31:0/3343524247 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5480039a0 0x7fc55c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 >> 172.21.15.31:0/3343524247 conn(0x7fc55c06abe0 msgr2=0x7fc55c0cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 shutdown_connections 2024-01-19T04:55:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:40.587+0000 7fc560ad0640 1 -- 172.21.15.31:0/3343524247 wait complete. 2024-01-19T04:55:40.597 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:55:40.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:55:40.597 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi137.front.sepia.ceph.com 2024-01-19T04:55:40.597 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-19T04:55:40.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:40.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:55:40.598 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:55:40.598 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:55:40.598 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-19T04:55:40.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:40.598 DEBUG:teuthology.orchestra.run.smithi137:> ip addr 2024-01-19T04:55:40.617 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7e brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f0 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7f brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f1 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:36 brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:40.619 INFO:teuthology.orchestra.run.smithi137.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:55:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:37 brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: inet 172.21.15.137/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-19T04:55:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft 42762sec preferred_lft 42762sec 2024-01-19T04:55:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::ec4:7aff:fe88:8137/64 scope link 2024-01-19T04:55:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:40.619 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-19T04:55:40.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:40.619 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:40.619 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:40.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add name ceph-brx type bridge 2024-01-19T04:55:40.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr flush dev ceph-brx 2024-01-19T04:55:40.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set ceph-brx up 2024-01-19T04:55:40.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-19T04:55:40.620 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:40.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:40.784 DEBUG:teuthology.orchestra.run.smithi137:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:55:40.812 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-01-19T04:55:40.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:40.814 DEBUG:teuthology.orchestra.run.smithi137:> route 2024-01-19T04:55:40.886 INFO:teuthology.orchestra.run.smithi137.stdout:Kernel IP routing table 2024-01-19T04:55:40.886 INFO:teuthology.orchestra.run.smithi137.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:55:40.887 INFO:teuthology.orchestra.run.smithi137.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-19T04:55:40.887 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-19T04:55:40.887 INFO:teuthology.orchestra.run.smithi137.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:55:40.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:40.887 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:40.887 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:40.887 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-19T04:55:40.887 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-19T04:55:40.887 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-19T04:55:40.887 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:41.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:41.192 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-01-19T04:55:41.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:41.206 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list-id 2024-01-19T04:55:41.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:41.261 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:41.261 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:41.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:41.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-19T04:55:41.261 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:41.346 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-19T04:55:41.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:41.346 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:41.346 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:41.346 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-19T04:55:41.346 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-19T04:55:41.346 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-19T04:55:41.346 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-19T04:55:41.346 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-19T04:55:41.347 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:41.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:41.441 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:41.441 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:41.441 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set brx.0 up 2024-01-19T04:55:41.441 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set dev brx.0 master ceph-brx 2024-01-19T04:55:41.441 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:41.484 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-19T04:55:41.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:41.484 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:41.539 INFO:teuthology.orchestra.run.smithi137.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-19T04:55:41.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:41.539 DEBUG:teuthology.orchestra.run.smithi137:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:41.594 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-01-19T04:55:41.663 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:41.718 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-01-19T04:55:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-4 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-5 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-6 2024-01-19T04:55:41.720 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-7 2024-01-19T04:55:41.721 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-01-19T04:55:41.721 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:41.721 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:41.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:41.722 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-01-19T04:55:41.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:55:41.776 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-19T04:55:41.818 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-01-19T04:55:41.845 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:41.900 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-4 2024-01-19T04:55:41.902 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-5 2024-01-19T04:55:41.903 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-6 2024-01-19T04:55:41.903 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-7 2024-01-19T04:55:41.903 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-01-19T04:55:41.903 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:41.903 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:41.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:41.903 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-01-19T04:55:41.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi137.stderr:2024-01-19T04:55:41.920+0000 7f9a8a238480 -1 init, newargv = 0x55a768e8d410 newargc=15 2024-01-19T04:55:41.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi137.stderr:ceph-fuse[43151]: starting ceph client 2024-01-19T04:55:41.924 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi137.stderr:ceph-fuse[43151]: starting fuse 2024-01-19T04:55:41.934 INFO:teuthology.orchestra.run.smithi137.stdout:41 2024-01-19T04:55:41.934 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:55:41.934 DEBUG:teuthology.orchestra.run.smithi137:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:55:41.934 DEBUG:teuthology.orchestra.run.smithi137:> import glob 2024-01-19T04:55:41.934 DEBUG:teuthology.orchestra.run.smithi137:> import re 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> import os 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> import subprocess 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> def _find_admin_socket(client_name): 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> files = glob.glob(asok_path) 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> # Given a non-glob path, it better be there 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> if "*" not in asok_path: 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> assert(len(files) == 1) 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> return files[0] 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> for f in files: 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:55:41.935 DEBUG:teuthology.orchestra.run.smithi137:> contents = proc_f.read() 2024-01-19T04:55:41.936 DEBUG:teuthology.orchestra.run.smithi137:> if mountpoint in contents: 2024-01-19T04:55:41.936 DEBUG:teuthology.orchestra.run.smithi137:> return f 2024-01-19T04:55:41.936 DEBUG:teuthology.orchestra.run.smithi137:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:55:41.936 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:41.936 DEBUG:teuthology.orchestra.run.smithi137:> print(_find_admin_socket("client.1")) 2024-01-19T04:55:41.936 DEBUG:teuthology.orchestra.run.smithi137:> ' 2024-01-19T04:55:42.053 INFO:teuthology.orchestra.run.smithi137.stdout:/var/run/ceph/ceph-client.1.43151.asok 2024-01-19T04:55:42.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:55:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:55:42.064 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43151.asok 2024-01-19T04:55:42.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.064 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43151.asok status 2024-01-19T04:55:42.202 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-01-19T04:55:42.202 INFO:teuthology.orchestra.run.smithi137.stdout: "metadata": { 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "entity_id": "1", 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": "smithi137", 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pid": "43151", 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "root": "/" 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_count": 0, 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_pinned_count": 0, 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 4529, 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "inst": { 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "name": { 2024-01-19T04:55:42.203 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "client", 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "num": 4529 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 3561329664 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 3561329664 2024-01-19T04:55:42.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "inst_str": "client.4529 192.168.0.1:0/3561329664", 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "addr_str": "192.168.0.1:0/3561329664", 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "inode_count": 1, 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_epoch": 6, 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch": 20, 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "blocklisted": false, 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs" 2024-01-19T04:55:42.205 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-01-19T04:55:42.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:55:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.324+0000 7ffb6f62b640 1 Processor -- start 2024-01-19T04:55:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.325+0000 7ffb6f62b640 1 -- start start 2024-01-19T04:55:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.325+0000 7ffb6f62b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb680b69d0 0x7ffb680cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.325+0000 7ffb6f62b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb6815ca20 0x7ffb680d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.325+0000 7ffb6f62b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6815c610 con 0x7ffb680d0940 2024-01-19T04:55:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.325+0000 7ffb6f62b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb680d0dd0 con 0x7ffb680b69d0 2024-01-19T04:55:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.325+0000 7ffb6f62b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6815ba00 con 0x7ffb6815ca20 2024-01-19T04:55:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.325+0000 7ffb6d3c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb680b69d0 0x7ffb680cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6d3c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb680b69d0 0x7ffb680cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53802/0 (socket says 172.21.15.31:53802) 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6dbc8640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffb680d0940 0x7ffb6815de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53716/0 (socket says 172.21.15.31:53716) 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/896956791 learned_addr learned my addr 172.21.15.31:0/896956791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6cbc6640 1 --2- 172.21.15.31:0/896956791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb6815ca20 0x7ffb680d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/896956791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb6815ca20 msgr2=0x7ffb680d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6d3c7640 1 --2- 172.21.15.31:0/896956791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb6815ca20 0x7ffb680d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/896956791 >> v1:172.21.15.31:6789/0 conn(0x7ffb680d0940 legacy=0x7ffb6815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb5ffff640 1 -- 172.21.15.31:0/896956791 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813591599 0 0) 0x7ffb6815c610 con 0x7ffb680d0940 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.326+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/896956791 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6815bce0 con 0x7ffb680b69d0 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb6d3c7640 1 --2- 172.21.15.31:0/896956791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680b69d0 0x7ffb680cb710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009870 tx=0x7ffb6002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd07b42eb57143bd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb5ffff640 1 -- 172.21.15.31:0/896956791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60030840 con 0x7ffb680b69d0 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb5ffff640 1 -- 172.21.15.31:0/896956791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb600309a0 con 0x7ffb680b69d0 2024-01-19T04:55:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb5ffff640 1 -- 172.21.15.31:0/896956791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60030d30 con 0x7ffb680b69d0 2024-01-19T04:55:42.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb6f62b640 1 -- 172.21.15.31:0/896956791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680b69d0 msgr2=0x7ffb680cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:42.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/896956791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680b69d0 0x7ffb680cb710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009870 tx=0x7ffb6002b980 comp rx=0 tx=0).stop 2024-01-19T04:55:42.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb6f62b640 1 -- 172.21.15.31:0/896956791 shutdown_connections 2024-01-19T04:55:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/896956791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb6815ca20 0x7ffb680d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/896956791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680b69d0 0x7ffb680cb710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.327+0000 7ffb6f62b640 1 -- 172.21.15.31:0/896956791 >> 172.21.15.31:0/896956791 conn(0x7ffb680d10c0 msgr2=0x7ffb680d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.328+0000 7ffb6f62b640 1 -- 172.21.15.31:0/896956791 shutdown_connections 2024-01-19T04:55:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.328+0000 7ffb6f62b640 1 -- 172.21.15.31:0/896956791 wait complete. 2024-01-19T04:55:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 Processor -- start 2024-01-19T04:55:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 -- start start 2024-01-19T04:55:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680d0940 0x7ffb680a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6815ca20 0x7ffb6806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb681f6160 0x7ffb6806cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68154d30 con 0x7ffb680d0940 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6806bf10 con 0x7ffb6815ca20 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.329+0000 7ffb6f62b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6806c1f0 con 0x7ffb681f6160 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680d0940 0x7ffb680a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680d0940 0x7ffb680a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53804/0 (socket says 172.21.15.31:53804) 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/2701813997 learned_addr learned my addr 172.21.15.31:0/2701813997 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6dbc8640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb681f6160 0x7ffb6806cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6cbc6640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6815ca20 0x7ffb6806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb681f6160 msgr2=0x7ffb6806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb681f6160 0x7ffb6806cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6815ca20 msgr2=0x7ffb6806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6815ca20 0x7ffb6806b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 -- 172.21.15.31:0/2701813997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600094d0 con 0x7ffb680d0940 2024-01-19T04:55:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6d3c7640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680d0940 0x7ffb680a92d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002bf10 tx=0x7ffb60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb5dffb640 1 -- 172.21.15.31:0/2701813997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60047070 con 0x7ffb680d0940 2024-01-19T04:55:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb5dffb640 1 -- 172.21.15.31:0/2701813997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60039a90 con 0x7ffb680d0940 2024-01-19T04:55:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6806c4d0 con 0x7ffb680d0940 2024-01-19T04:55:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.330+0000 7ffb5dffb640 1 -- 172.21.15.31:0/2701813997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60039e40 con 0x7ffb680d0940 2024-01-19T04:55:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.331+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6806ca90 con 0x7ffb680d0940 2024-01-19T04:55:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.332+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb68074a80 con 0x7ffb680d0940 2024-01-19T04:55:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.332+0000 7ffb5dffb640 1 -- 172.21.15.31:0/2701813997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ffb6004c410 con 0x7ffb680d0940 2024-01-19T04:55:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.332+0000 7ffb5dffb640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffb4c06aed0 0x7ffb4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.333+0000 7ffb5dffb640 1 -- 172.21.15.31:0/2701813997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb6003e070 con 0x7ffb680d0940 2024-01-19T04:55:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.333+0000 7ffb6cbc6640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffb4c06aed0 0x7ffb4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.333+0000 7ffb6cbc6640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffb4c06aed0 0x7ffb4c06d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb680cb990 tx=0x7ffb58009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.337+0000 7ffb5dffb640 1 -- 172.21.15.31:0/2701813997 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ffb60076e20 con 0x7ffb680d0940 2024-01-19T04:55:42.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.554+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb680696e0 con 0x7ffb680d0940 2024-01-19T04:55:42.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.554+0000 7ffb5dffb640 1 -- 172.21.15.31:0/2701813997 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb6007a440 con 0x7ffb680d0940 2024-01-19T04:55:42.554 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:55:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffb4c06aed0 msgr2=0x7ffb4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffb4c06aed0 0x7ffb4c06d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb680cb990 tx=0x7ffb58009190 comp rx=0 tx=0).stop 2024-01-19T04:55:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680d0940 msgr2=0x7ffb680a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680d0940 0x7ffb680a92d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002bf10 tx=0x7ffb60004730 comp rx=0 tx=0).stop 2024-01-19T04:55:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 shutdown_connections 2024-01-19T04:55:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffb4c06aed0 0x7ffb4c06d390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffb681f6160 0x7ffb6806cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6815ca20 0x7ffb6806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 --2- 172.21.15.31:0/2701813997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb680d0940 0x7ffb680a92d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 >> 172.21.15.31:0/2701813997 conn(0x7ffb680d10c0 msgr2=0x7ffb680cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 shutdown_connections 2024-01-19T04:55:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:42.556+0000 7ffb6f62b640 1 -- 172.21.15.31:0/2701813997 wait complete. 2024-01-19T04:55:42.566 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:55:42.566 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:55:42.566 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi137.front.sepia.ceph.com 2024-01-19T04:55:42.566 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-19T04:55:42.566 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-19T04:55:42.567 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:55:42.567 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:55:42.567 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:55:42.567 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-19T04:55:42.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.567 DEBUG:teuthology.orchestra.run.smithi137:> ip addr 2024-01-19T04:55:42.582 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7e brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f0 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7f brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f1 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:55:42.583 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:36 brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:37 brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: inet 172.21.15.137/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft 42760sec preferred_lft 42760sec 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::ec4:7aff:fe88:8137/64 scope link 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether d2:f1:39:dd:74:ad brd ff:ff:ff:ff:ff:ff 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::309c:a3ff:fe7c:33ca/64 scope link tentative 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether d2:f1:39:dd:74:ad brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:42.584 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::d0f1:39ff:fedd:74ad/64 scope link tentative 2024-01-19T04:55:42.585 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:42.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.585 DEBUG:teuthology.orchestra.run.smithi137:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:55:42.649 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-01-19T04:55:42.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.651 DEBUG:teuthology.orchestra.run.smithi137:> route 2024-01-19T04:55:42.707 INFO:teuthology.orchestra.run.smithi137.stdout:Kernel IP routing table 2024-01-19T04:55:42.707 INFO:teuthology.orchestra.run.smithi137.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:55:42.707 INFO:teuthology.orchestra.run.smithi137.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-19T04:55:42.707 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-19T04:55:42.707 INFO:teuthology.orchestra.run.smithi137.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:55:42.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.707 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:42.707 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:42.707 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-19T04:55:42.708 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-19T04:55:42.708 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-19T04:55:42.708 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:42.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.802 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-01-19T04:55:42.817 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-19T04:55:42.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.817 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list-id 2024-01-19T04:55:42.871 INFO:teuthology.orchestra.run.smithi137.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-19T04:55:42.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.872 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:42.872 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:42.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-19T04:55:42.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-19T04:55:42.872 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:42.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:42.955 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-19T04:55:43.024 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:43.024 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:43.024 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 06:be:df:3c:13:3a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::4be:dfff:fe3c:133a/64 scope link tentative 2024-01-19T04:55:43.025 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:43.026 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-19T04:55:43.099 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:55:43.099 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:55:43.099 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:55:43.099 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.099 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-01-19T04:55:43.099 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.099 INFO:teuthology.orchestra.run.smithi137.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:55:43.100 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 06:be:df:3c:13:3a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-19T04:55:43.100 INFO:teuthology.orchestra.run.smithi137.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-19T04:55:43.100 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.100 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::4be:dfff:fe3c:133a/64 scope link tentative 2024-01-19T04:55:43.100 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:55:43.101 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-19T04:55:43.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:43.101 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:43.101 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:43.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-19T04:55:43.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-19T04:55:43.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-19T04:55:43.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-19T04:55:43.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-19T04:55:43.102 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:43.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:43.229 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-01-19T04:55:43.229 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-01-19T04:55:43.229 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set brx.1 up 2024-01-19T04:55:43.229 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set dev brx.1 master ceph-brx 2024-01-19T04:55:43.229 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-01-19T04:55:43.276 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-01-19T04:55:43.276 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:43.276 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-19T04:55:43.335 INFO:teuthology.orchestra.run.smithi137.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-19T04:55:43.336 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:55:43.336 DEBUG:teuthology.orchestra.run.smithi137:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-19T04:55:43.397 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-01-19T04:55:43.470 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:43.531 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-01-19T04:55:43.531 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-01-19T04:55:43.531 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-01-19T04:55:43.531 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-01-19T04:55:43.531 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-01-19T04:55:43.531 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-01-19T04:55:43.532 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:43.533 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-4 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-5 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-6 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-7 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-19T04:55:43.534 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-19T04:55:43.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:43.535 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-01-19T04:55:43.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:55:43.550 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:55:43.589 INFO:teuthology.orchestra.run.smithi137.stdout:41 2024-01-19T04:55:43.589 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-01-19T04:55:43.589 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-01-19T04:55:43.630 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-01-19T04:55:43.656 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:55:43.714 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-01-19T04:55:43.714 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-01-19T04:55:43.715 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-4 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-5 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-6 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph/osd/ceph-7 2024-01-19T04:55:43.716 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-01-19T04:55:43.717 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:43.717 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:55:43.717 INFO:teuthology.orchestra.run.smithi137.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:43.717 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-19T04:55:43.717 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-19T04:55:43.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:43.717 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-01-19T04:55:43.718 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi137.stderr:2024-01-19T04:55:43.715+0000 7f1484845480 -1 init, newargv = 0x55f295a6d4a0 newargc=15 2024-01-19T04:55:43.718 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi137.stderr:ceph-fuse[43604]: starting ceph client 2024-01-19T04:55:43.719 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi137.stderr:ceph-fuse[43604]: starting fuse 2024-01-19T04:55:43.734 INFO:teuthology.orchestra.run.smithi137.stdout:41 2024-01-19T04:55:43.734 INFO:teuthology.orchestra.run.smithi137.stdout:44 2024-01-19T04:55:43.734 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-01-19T04:55:43.734 DEBUG:teuthology.orchestra.run.smithi137:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> import glob 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> import re 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> import os 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> import subprocess 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> def _find_admin_socket(client_name): 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> files = glob.glob(asok_path) 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> # Given a non-glob path, it better be there 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> if "*" not in asok_path: 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> assert(len(files) == 1) 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> return files[0] 2024-01-19T04:55:43.735 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> for f in files: 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> contents = proc_f.read() 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> if mountpoint in contents: 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> return f 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> print(_find_admin_socket("client.3")) 2024-01-19T04:55:43.736 DEBUG:teuthology.orchestra.run.smithi137:> ' 2024-01-19T04:55:43.853 INFO:teuthology.orchestra.run.smithi137.stdout:/var/run/ceph/ceph-client.3.43604.asok 2024-01-19T04:55:43.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-19T04:55:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:55:43.863 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43604.asok 2024-01-19T04:55:43.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:43.863 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43604.asok status 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "metadata": { 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "entity_id": "3", 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": "smithi137", 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "pid": "43604", 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: "root": "/" 2024-01-19T04:55:43.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_count": 0, 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_pinned_count": 0, 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 4660, 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "inst": { 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "name": { 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "client", 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "num": 4660 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.2:0", 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 2155170993 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-01-19T04:55:44.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.2:0", 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 2155170993 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "inst_str": "client.4660 192.168.0.2:0/2155170993", 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "addr_str": "192.168.0.2:0/2155170993", 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "inode_count": 1, 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_epoch": 6, 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch": 20, 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "blocklisted": false, 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs" 2024-01-19T04:55:44.001 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-01-19T04:55:44.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.007 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:55:44.007 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:44.026 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:55:44.026 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:44.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:44.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.099 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:55:44.099 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.2 2024-01-19T04:55:44.156 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:55:44.156 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-01-19T04:55:44.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-19T04:55:44.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.229 DEBUG:teuthology.orchestra.run.smithi137:> stat --file-system '--printf=%T 2024-01-19T04:55:44.229 DEBUG:teuthology.orchestra.run.smithi137:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:44.245 INFO:teuthology.orchestra.run.smithi137.stdout:fuseblk 2024-01-19T04:55:44.245 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:44.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.245 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:44.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.312 DEBUG:teuthology.orchestra.run.smithi137:> stat --file-system '--printf=%T 2024-01-19T04:55:44.312 DEBUG:teuthology.orchestra.run.smithi137:> ' -- /home/ubuntu/cephtest/mnt.3 2024-01-19T04:55:44.367 INFO:teuthology.orchestra.run.smithi137.stdout:fuseblk 2024-01-19T04:55:44.367 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-01-19T04:55:44.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:44.367 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-19T04:55:44.434 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-19T04:55:44.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:44.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.561+0000 7f67dfd90640 1 Processor -- start 2024-01-19T04:55:44.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.561+0000 7f67dfd90640 1 -- start start 2024-01-19T04:55:44.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67dfd90640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815f0c0 0x7f67d815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67dfd90640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d806bf30 con 0x7f67d815eb70 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67dfd90640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d80cb9e0 con 0x7f67d815d8a0 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67dfd90640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d815fa30 con 0x7f67d815f0c0 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67dd32b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67d815eb70 0x7f67d80cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53730/0 (socket says 172.21.15.31:53730) 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67dd32b640 1 -- 172.21.15.31:0/2155677271 learned_addr learned my addr 172.21.15.31:0/2155677271 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67de32d640 1 --2- 172.21.15.31:0/2155677271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815f0c0 0x7f67d815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.562+0000 7f67dcb2a640 1 -- 172.21.15.31:0/2155677271 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493908770 0 0) 0x7f67d806bf30 con 0x7f67d815eb70 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.563+0000 7f67dcb2a640 1 -- 172.21.15.31:0/2155677271 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c4003520 con 0x7f67d815eb70 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.563+0000 7f67dcb2a640 1 -- 172.21.15.31:0/2155677271 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617609314 0 0) 0x7f67d80cb9e0 con 0x7f67d815d8a0 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.563+0000 7f67dcb2a640 1 -- 172.21.15.31:0/2155677271 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d806bf30 con 0x7f67d815d8a0 2024-01-19T04:55:44.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.563+0000 7f67de32d640 1 -- 172.21.15.31:0/2155677271 >> v1:172.21.15.137:6789/0 conn(0x7f67d815d8a0 legacy=0x7f67d806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.563+0000 7f67de32d640 1 -- 172.21.15.31:0/2155677271 >> v1:172.21.15.31:6789/0 conn(0x7f67d815eb70 legacy=0x7f67d80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.563+0000 7f67de32d640 1 -- 172.21.15.31:0/2155677271 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80cbcc0 con 0x7f67d815f0c0 2024-01-19T04:55:44.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.563+0000 7f67de32d640 1 --2- 172.21.15.31:0/2155677271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815f0c0 0x7f67d815f4f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009dc0 tx=0x7f67d402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fe344b243d9f7ee server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dcb2a640 1 -- 172.21.15.31:0/2155677271 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d402cc40 con 0x7f67d815f0c0 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dcb2a640 1 -- 172.21.15.31:0/2155677271 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d402cda0 con 0x7f67d815f0c0 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dcb2a640 1 -- 172.21.15.31:0/2155677271 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4034660 con 0x7f67d815f0c0 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dfd90640 1 -- 172.21.15.31:0/2155677271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815f0c0 msgr2=0x7f67d815f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dfd90640 1 --2- 172.21.15.31:0/2155677271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815f0c0 0x7f67d815f4f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009dc0 tx=0x7f67d402be90 comp rx=0 tx=0).stop 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dfd90640 1 -- 172.21.15.31:0/2155677271 shutdown_connections 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dfd90640 1 --2- 172.21.15.31:0/2155677271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815f0c0 0x7f67d815f4f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dfd90640 1 -- 172.21.15.31:0/2155677271 >> 172.21.15.31:0/2155677271 conn(0x7f67d806abe0 msgr2=0x7f67d8066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dfd90640 1 -- 172.21.15.31:0/2155677271 shutdown_connections 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.564+0000 7f67dfd90640 1 -- 172.21.15.31:0/2155677271 wait complete. 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.565+0000 7f67dfd90640 1 Processor -- start 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67dfd90640 1 -- start start 2024-01-19T04:55:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67dfd90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d815d8a0 0x7f67d81571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67dfd90640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815eb70 0x7f67d80c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67dfd90640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67d815f0c0 0x7f67d80b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67dfd90640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80b4140 con 0x7f67d815d8a0 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67dfd90640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80c6140 con 0x7f67d815f0c0 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67dfd90640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80c6420 con 0x7f67d815eb70 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67ddb2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d815d8a0 0x7f67d81571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67ddb2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d815d8a0 0x7f67d81571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53806/0 (socket says 172.21.15.31:53806) 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.566+0000 7f67ddb2c640 1 -- 172.21.15.31:0/1268747124 learned_addr learned my addr 172.21.15.31:0/1268747124 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67dd32b640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815eb70 0x7f67d80c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67de32d640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67d815f0c0 0x7f67d80b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ddb2c640 1 -- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815eb70 msgr2=0x7f67d80c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ddb2c640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815eb70 0x7f67d80c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ddb2c640 1 -- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67d815f0c0 msgr2=0x7f67d80b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ddb2c640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67d815f0c0 0x7f67d80b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ddb2c640 1 -- 172.21.15.31:0/1268747124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4009a70 con 0x7f67d815d8a0 2024-01-19T04:55:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ddb2c640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d815d8a0 0x7f67d81571f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67d000c500 tx=0x7f67d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ce7fc640 1 -- 172.21.15.31:0/1268747124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d00052d0 con 0x7f67d815d8a0 2024-01-19T04:55:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ce7fc640 1 -- 172.21.15.31:0/1268747124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0005450 con 0x7f67d815d8a0 2024-01-19T04:55:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d80c6700 con 0x7f67d815d8a0 2024-01-19T04:55:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67ce7fc640 1 -- 172.21.15.31:0/1268747124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0012870 con 0x7f67d815d8a0 2024-01-19T04:55:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.567+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d80ca150 con 0x7f67d815d8a0 2024-01-19T04:55:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.568+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d80696e0 con 0x7f67d815d8a0 2024-01-19T04:55:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.568+0000 7f67ce7fc640 1 -- 172.21.15.31:0/1268747124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f67d00129d0 con 0x7f67d815d8a0 2024-01-19T04:55:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.569+0000 7f67ce7fc640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f67bc06aed0 0x7f67bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.569+0000 7f67ce7fc640 1 -- 172.21.15.31:0/1268747124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67d00892a0 con 0x7f67d815d8a0 2024-01-19T04:55:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.569+0000 7f67dd32b640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f67bc06aed0 0x7f67bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.569+0000 7f67dd32b640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f67bc06aed0 0x7f67bc06d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67d81f6a30 tx=0x7f67c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.573+0000 7f67ce7fc640 1 -- 172.21.15.31:0/1268747124 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f67d0057c10 con 0x7f67d815d8a0 2024-01-19T04:55:44.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.756+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d80b71e0 con 0x7f67d815d8a0 2024-01-19T04:55:44.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.757+0000 7f67ce7fc640 1 -- 172.21.15.31:0/1268747124 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f67d00577a0 con 0x7f67d815d8a0 2024-01-19T04:55:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:44.757 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f67bc06aed0 msgr2=0x7f67bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f67bc06aed0 0x7f67bc06d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67d81f6a30 tx=0x7f67c8009040 comp rx=0 tx=0).stop 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d815d8a0 msgr2=0x7f67d81571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d815d8a0 0x7f67d81571f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67d000c500 tx=0x7f67d000c9c0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 shutdown_connections 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f67bc06aed0 0x7f67bc06d390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67d815f0c0 0x7f67d80b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67d815eb70 0x7f67d80c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 --2- 172.21.15.31:0/1268747124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d815d8a0 0x7f67d81571f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 >> 172.21.15.31:0/1268747124 conn(0x7f67d806abe0 msgr2=0x7f67d80b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 shutdown_connections 2024-01-19T04:55:44.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.759+0000 7f67dfd90640 1 -- 172.21.15.31:0/1268747124 wait complete. 2024-01-19T04:55:44.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:44.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.874+0000 7f9d2bf5f640 1 Processor -- start 2024-01-19T04:55:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.874+0000 7f9d2bf5f640 1 -- start start 2024-01-19T04:55:44.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d2bf5f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 0x7f9d240d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d2bf5f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d240cef10 con 0x7f9d240d1fe0 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d2bf5f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d240ce300 con 0x7f9d2415d700 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d2bf5f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240ce5e0 con 0x7f9d240d11b0 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d2a4fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9d240d1fe0 0x7f9d240b2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53744/0 (socket says 172.21.15.31:53744) 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d2a4fc640 1 -- 172.21.15.31:0/100075054 learned_addr learned my addr 172.21.15.31:0/100075054 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d294fa640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 0x7f9d240d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.875+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673563688 0 0) 0x7f9d240cef10 con 0x7f9d240d1fe0 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d0c003520 con 0x7f9d240d1fe0 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456222102 0 0) 0x7f9d240ce300 con 0x7f9d2415d700 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d240cef10 con 0x7f9d2415d700 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539763067 0 0) 0x7f9d0c003520 con 0x7f9d240d1fe0 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 msgr2=0x7f9d240d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 0x7f9d240d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 >> v1:172.21.15.137:6789/0 conn(0x7f9d2415d700 legacy=0x7f9d240d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240ce8c0 con 0x7f9d240d1fe0 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f9d1c001670 con 0x7f9d240d1fe0 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 >> v1:172.21.15.31:6789/0 conn(0x7f9d240d1fe0 legacy=0x7f9d240b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 0x7f9d0c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d0c004d00 0x7f9d0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240ce5e0 con 0x7f9d0c0040e0 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c00bac0 con 0x7f9d0c004d00 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c00bc80 con 0x7f9d240d11b0 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d294fa640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d2a4fc640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d0c004d00 0x7f9d0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d29cfb640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 0x7f9d0c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d294fa640 1 -- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 msgr2=0x7f9d0c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d294fa640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d240d11b0 0x7f9d0c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d294fa640 1 -- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d0c004d00 msgr2=0x7f9d0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d294fa640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d0c004d00 0x7f9d0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.876+0000 7f9d294fa640 1 -- 172.21.15.31:0/100075054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c003520 con 0x7f9d0c0040e0 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.877+0000 7f9d294fa640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d0c0047c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009aa0 tx=0x7f9d0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b373db62297e06 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.877+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1802c600 con 0x7f9d0c0040e0 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.877+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1802c760 con 0x7f9d0c0040e0 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.877+0000 7f9d28cf9640 1 -- 172.21.15.31:0/100075054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1802cb10 con 0x7f9d0c0040e0 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.878+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/100075054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 msgr2=0x7f9d0c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.878+0000 7f9d2bf5f640 1 --2- 172.21.15.31:0/100075054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d0c0047c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009aa0 tx=0x7f9d0c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.878+0000 7f9d2a4fc640 1 -- 172.21.15.31:0/100075054 reap_dead start 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.878+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/100075054 shutdown_connections 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.878+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/100075054 >> 172.21.15.31:0/100075054 conn(0x7f9d240d32f0 msgr2=0x7f9d240d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.878+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/100075054 shutdown_connections 2024-01-19T04:55:44.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.878+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/100075054 wait complete. 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.879+0000 7f9d2bf5f640 1 Processor -- start 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.879+0000 7f9d2bf5f640 1 -- start start 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d240a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d2409e6f0 0x7f9d240cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d2406c620 0x7f9d24159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2409f400 con 0x7f9d0c0040e0 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240ca8d0 con 0x7f9d2409e6f0 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240cb2a0 con 0x7f9d2406c620 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d240a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d240a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53810/0 (socket says 172.21.15.31:53810) 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 -- 172.21.15.31:0/1285239244 learned_addr learned my addr 172.21.15.31:0/1285239244 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d294fa640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d2409e6f0 0x7f9d240cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2a4fc640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d2406c620 0x7f9d24159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 -- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d2406c620 msgr2=0x7f9d24159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d2406c620 0x7f9d24159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 -- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d2409e6f0 msgr2=0x7f9d240cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d2409e6f0 0x7f9d240cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 -- 172.21.15.31:0/1285239244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2409eb70 con 0x7f9d0c0040e0 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2a4fc640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d2406c620 0x7f9d24159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d294fa640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d2409e6f0 0x7f9d240cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d29cfb640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d240a08b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400fbb0 tx=0x7f9d1400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d227fc640 1 -- 172.21.15.31:0/1285239244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400dc30 con 0x7f9d0c0040e0 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d227fc640 1 -- 172.21.15.31:0/1285239244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400ddb0 con 0x7f9d0c0040e0 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2409ee40 con 0x7f9d0c0040e0 2024-01-19T04:55:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d227fc640 1 -- 172.21.15.31:0/1285239244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14003bb0 con 0x7f9d0c0040e0 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.880+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d240ca560 con 0x7f9d0c0040e0 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.881+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d240ca3a0 con 0x7f9d0c0040e0 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.882+0000 7f9d227fc640 1 -- 172.21.15.31:0/1285239244 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9d14003d10 con 0x7f9d0c0040e0 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.882+0000 7f9d227fc640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1006af00 0x7f9d1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.882+0000 7f9d227fc640 1 -- 172.21.15.31:0/1285239244 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d14089a20 con 0x7f9d0c0040e0 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.882+0000 7f9d294fa640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1006af00 0x7f9d1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.883+0000 7f9d294fa640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1006af00 0x7f9d1006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d18036000 tx=0x7f9d18030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:44.886+0000 7f9d227fc640 1 -- 172.21.15.31:0/1285239244 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9d140548e0 con 0x7f9d0c0040e0 2024-01-19T04:55:45.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.063+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d24074900 con 0x7f9d0c0040e0 2024-01-19T04:55:45.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.063+0000 7f9d227fc640 1 -- 172.21.15.31:0/1285239244 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d14057f00 con 0x7f9d0c0040e0 2024-01-19T04:55:45.063 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1006af00 msgr2=0x7f9d1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1006af00 0x7f9d1006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d18036000 tx=0x7f9d18030000 comp rx=0 tx=0).stop 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 msgr2=0x7f9d240a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d240a08b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400fbb0 tx=0x7f9d1400d520 comp rx=0 tx=0).stop 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 shutdown_connections 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1006af00 0x7f9d1006d3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d2406c620 0x7f9d24159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d2409e6f0 0x7f9d240cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 --2- 172.21.15.31:0/1285239244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d0c0040e0 0x7f9d240a08b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 >> 172.21.15.31:0/1285239244 conn(0x7f9d240d32f0 msgr2=0x7f9d2415c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 shutdown_connections 2024-01-19T04:55:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.065+0000 7f9d2bf5f640 1 -- 172.21.15.31:0/1285239244 wait complete. 2024-01-19T04:55:45.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.179+0000 7fc463bb0640 1 Processor -- start 2024-01-19T04:55:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.180+0000 7fc463bb0640 1 -- start start 2024-01-19T04:55:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.180+0000 7fc463bb0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d18e0 0x7fc45c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.180+0000 7fc463bb0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc45c0d0ef0 0x7fc45c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.180+0000 7fc463bb0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45c15dee0 con 0x7fc45c0d3e50 2024-01-19T04:55:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.180+0000 7fc463bb0640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c15e1c0 con 0x7fc45c0d0ef0 2024-01-19T04:55:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.180+0000 7fc463bb0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c15e4a0 con 0x7fc45c0d18e0 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46194c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d18e0 0x7fc45c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46214d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc45c0d0ef0 0x7fc45c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46114b640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fc45c0d3e50 0x7fc45c0d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:60716/0 (socket says 172.21.15.31:60716) 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46114b640 1 -- 172.21.15.31:0/3232346965 learned_addr learned my addr 172.21.15.31:0/3232346965 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46194c640 1 -- 172.21.15.31:0/3232346965 >> v2:172.21.15.137:3300/0 conn(0x7fc45c0d0ef0 msgr2=0x7fc45c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46194c640 1 --2- 172.21.15.31:0/3232346965 >> v2:172.21.15.137:3300/0 conn(0x7fc45c0d0ef0 0x7fc45c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46094a640 1 -- 172.21.15.31:0/3232346965 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780809539 0 0) 0x7fc45c15dee0 con 0x7fc45c0d3e50 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46194c640 1 -- 172.21.15.31:0/3232346965 >> v1:172.21.15.137:6789/0 conn(0x7fc45c0d3e50 legacy=0x7fc45c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.181+0000 7fc46194c640 1 -- 172.21.15.31:0/3232346965 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c15e780 con 0x7fc45c0d18e0 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.182+0000 7fc46194c640 1 --2- 172.21.15.31:0/3232346965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d18e0 0x7fc45c15d860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4580097a0 tx=0x7fc45802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b487c28b27ce238c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.182+0000 7fc46094a640 1 -- 172.21.15.31:0/3232346965 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458031830 con 0x7fc45c0d18e0 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.182+0000 7fc46094a640 1 -- 172.21.15.31:0/3232346965 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc458031990 con 0x7fc45c0d18e0 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.182+0000 7fc46094a640 1 -- 172.21.15.31:0/3232346965 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458031d20 con 0x7fc45c0d18e0 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 -- 172.21.15.31:0/3232346965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d18e0 msgr2=0x7fc45c15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 --2- 172.21.15.31:0/3232346965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d18e0 0x7fc45c15d860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4580097a0 tx=0x7fc45802b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 -- 172.21.15.31:0/3232346965 shutdown_connections 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 --2- 172.21.15.31:0/3232346965 >> v2:172.21.15.137:3300/0 conn(0x7fc45c0d0ef0 0x7fc45c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 --2- 172.21.15.31:0/3232346965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d18e0 0x7fc45c15d860 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 -- 172.21.15.31:0/3232346965 >> 172.21.15.31:0/3232346965 conn(0x7fc45c09c600 msgr2=0x7fc45c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:45.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 -- 172.21.15.31:0/3232346965 shutdown_connections 2024-01-19T04:55:45.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.183+0000 7fc463bb0640 1 -- 172.21.15.31:0/3232346965 wait complete. 2024-01-19T04:55:45.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.184+0000 7fc463bb0640 1 Processor -- start 2024-01-19T04:55:45.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.184+0000 7fc463bb0640 1 -- start start 2024-01-19T04:55:45.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc463bb0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d0ef0 0x7fc45c0b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc463bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c0d18e0 0x7fc45c0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc463bb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45c0d3e50 0x7fc45c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc463bb0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c14ab40 con 0x7fc45c0d3e50 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc463bb0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0b8dc0 con 0x7fc45c0d18e0 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc463bb0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0b29d0 con 0x7fc45c0d0ef0 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45c0d3e50 0x7fc45c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45c0d3e50 0x7fc45c0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53818/0 (socket says 172.21.15.31:53818) 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 -- 172.21.15.31:0/2796792344 learned_addr learned my addr 172.21.15.31:0/2796792344 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46194c640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d0ef0 0x7fc45c0b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46114b640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c0d18e0 0x7fc45c0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 -- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d0ef0 msgr2=0x7fc45c0b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d0ef0 0x7fc45c0b86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 -- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c0d18e0 msgr2=0x7fc45c0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c0d18e0 0x7fc45c0b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 -- 172.21.15.31:0/2796792344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc458009400 con 0x7fc45c0d3e50 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc46214d640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45c0d3e50 0x7fc45c0b3cd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc45400efc0 tx=0x7fc45400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc4527fc640 1 -- 172.21.15.31:0/2796792344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454014480 con 0x7fc45c0d3e50 2024-01-19T04:55:45.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc4527fc640 1 -- 172.21.15.31:0/2796792344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45400f040 con 0x7fc45c0d3e50 2024-01-19T04:55:45.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c0b2cb0 con 0x7fc45c0d3e50 2024-01-19T04:55:45.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.185+0000 7fc4527fc640 1 -- 172.21.15.31:0/2796792344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454014820 con 0x7fc45c0d3e50 2024-01-19T04:55:45.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.186+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c0b3270 con 0x7fc45c0d3e50 2024-01-19T04:55:45.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.186+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c074900 con 0x7fc45c0d3e50 2024-01-19T04:55:45.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.187+0000 7fc4527fc640 1 -- 172.21.15.31:0/2796792344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc454014980 con 0x7fc45c0d3e50 2024-01-19T04:55:45.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.187+0000 7fc4527fc640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc44006af00 0x7fc44006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.187+0000 7fc4527fc640 1 -- 172.21.15.31:0/2796792344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc45408a110 con 0x7fc45c0d3e50 2024-01-19T04:55:45.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.188+0000 7fc46194c640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc44006af00 0x7fc44006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.188+0000 7fc46194c640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc44006af00 0x7fc44006d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc458036000 tx=0x7fc45803e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.192+0000 7fc4527fc640 1 -- 172.21.15.31:0/2796792344 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc454054fd0 con 0x7fc45c0d3e50 2024-01-19T04:55:45.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.368+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc45c0c9f10 con 0x7fc45c0d3e50 2024-01-19T04:55:45.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.368+0000 7fc4527fc640 1 -- 172.21.15.31:0/2796792344 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4540585f0 con 0x7fc45c0d3e50 2024-01-19T04:55:45.368 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc44006af00 msgr2=0x7fc44006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc44006af00 0x7fc44006d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc458036000 tx=0x7fc45803e040 comp rx=0 tx=0).stop 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45c0d3e50 msgr2=0x7fc45c0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45c0d3e50 0x7fc45c0b3cd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc45400efc0 tx=0x7fc45400c490 comp rx=0 tx=0).stop 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 shutdown_connections 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc44006af00 0x7fc44006d3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc45c0d3e50 0x7fc45c0b3cd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c0d18e0 0x7fc45c0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 --2- 172.21.15.31:0/2796792344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc45c0d0ef0 0x7fc45c0b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 >> 172.21.15.31:0/2796792344 conn(0x7fc45c09c600 msgr2=0x7fc45c0b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 shutdown_connections 2024-01-19T04:55:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.370+0000 7fc463bb0640 1 -- 172.21.15.31:0/2796792344 wait complete. 2024-01-19T04:55:45.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.485+0000 7f0050a2f640 1 Processor -- start 2024-01-19T04:55:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.485+0000 7f0050a2f640 1 -- start start 2024-01-19T04:55:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f0050a2f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f004c0d18e0 0x7f004c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f0050a2f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f0050a2f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c0d2200 con 0x7f004c0d0ef0 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f0050a2f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c15dee0 con 0x7f004c0d18e0 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f0050a2f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c15e1c0 con 0x7f004c0d3e50 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f004a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f004c0d18e0 0x7f004c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f004a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f004c0d18e0 0x7f004c15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53822/0 (socket says 172.21.15.31:53822) 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f004a59c640 1 -- 172.21.15.31:0/635143696 learned_addr learned my addr 172.21.15.31:0/635143696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f004a59c640 1 -- 172.21.15.31:0/635143696 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 msgr2=0x7f004c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f004a59c640 1 --2- 172.21.15.31:0/635143696 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f004a59c640 1 -- 172.21.15.31:0/635143696 >> v1:172.21.15.137:6789/0 conn(0x7f004c0d0ef0 legacy=0x7f004c161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:45.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.486+0000 7f004a59c640 1 -- 172.21.15.31:0/635143696 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c15e4a0 con 0x7f004c0d18e0 2024-01-19T04:55:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f004a59c640 1 --2- 172.21.15.31:0/635143696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c15d860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f003c009870 tx=0x7f003c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a68209a6ac8d8e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0049d9b640 1 -- 172.21.15.31:0/635143696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c031640 con 0x7f004c0d18e0 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0049d9b640 1 -- 172.21.15.31:0/635143696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c0317a0 con 0x7f004c0d18e0 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0049d9b640 1 -- 172.21.15.31:0/635143696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c031b30 con 0x7f004c0d18e0 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 -- 172.21.15.31:0/635143696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 msgr2=0x7f004c15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 --2- 172.21.15.31:0/635143696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c15d860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f003c009870 tx=0x7f003c02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 -- 172.21.15.31:0/635143696 shutdown_connections 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 --2- 172.21.15.31:0/635143696 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 --2- 172.21.15.31:0/635143696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c15d860 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 -- 172.21.15.31:0/635143696 >> 172.21.15.31:0/635143696 conn(0x7f004c09c600 msgr2=0x7f004c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 -- 172.21.15.31:0/635143696 shutdown_connections 2024-01-19T04:55:45.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.487+0000 7f0050a2f640 1 -- 172.21.15.31:0/635143696 wait complete. 2024-01-19T04:55:45.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.489+0000 7f0050a2f640 1 Processor -- start 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.489+0000 7f0050a2f640 1 -- start start 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0050a2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f004c0d0ef0 0x7f004c1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0050a2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c0b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0050a2f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0050a2f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0cb0a0 con 0x7f004c0d18e0 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0050a2f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0b87c0 con 0x7f004c0d0ef0 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0050a2f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c156d50 con 0x7f004c0d3e50 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c0b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c0b7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53824/0 (socket says 172.21.15.31:53824) 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 -- 172.21.15.31:0/3317891428 learned_addr learned my addr 172.21.15.31:0/3317891428 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f004ad9d640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f004a59c640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f004c0d0ef0 0x7f004c1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 -- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 msgr2=0x7f004c0b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 -- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f004c0d0ef0 msgr2=0x7f004c1f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f004c0d0ef0 0x7f004c1f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 -- 172.21.15.31:0/3317891428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0094d0 con 0x7f004c0d18e0 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f004ad9d640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f004a59c640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f004c0d0ef0 0x7f004c1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0041d9b640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c0b7d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f003400b410 tx=0x7f003400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f00437fe640 1 -- 172.21.15.31:0/3317891428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034011830 con 0x7f004c0d18e0 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f00437fe640 1 -- 172.21.15.31:0/3317891428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00340119b0 con 0x7f004c0d18e0 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c0ca490 con 0x7f004c0d18e0 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.490+0000 7f00437fe640 1 -- 172.21.15.31:0/3317891428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034011d60 con 0x7f004c0d18e0 2024-01-19T04:55:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.491+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c0caa50 con 0x7f004c0d18e0 2024-01-19T04:55:45.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.492+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c0cefd0 con 0x7f004c0d18e0 2024-01-19T04:55:45.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.492+0000 7f00437fe640 1 -- 172.21.15.31:0/3317891428 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f003401a400 con 0x7f004c0d18e0 2024-01-19T04:55:45.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.492+0000 7f00437fe640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f002c06aed0 0x7f002c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.492+0000 7f00437fe640 1 -- 172.21.15.31:0/3317891428 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00340897a0 con 0x7f004c0d18e0 2024-01-19T04:55:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.493+0000 7f004a59c640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f002c06aed0 0x7f002c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.493+0000 7f004a59c640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f002c06aed0 0x7f002c06d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f003c036000 tx=0x7f003c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.497+0000 7f00437fe640 1 -- 172.21.15.31:0/3317891428 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f004c0cefd0 con 0x7f004c0d18e0 2024-01-19T04:55:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.675+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f004c0cefd0 con 0x7f004c0d18e0 2024-01-19T04:55:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.675+0000 7f00437fe640 1 -- 172.21.15.31:0/3317891428 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f004c0cefd0 con 0x7f004c0d18e0 2024-01-19T04:55:45.675 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:45.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f002c06aed0 msgr2=0x7f002c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f002c06aed0 0x7f002c06d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f003c036000 tx=0x7f003c03c040 comp rx=0 tx=0).stop 2024-01-19T04:55:45.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 msgr2=0x7f004c0b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c0b7d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f003400b410 tx=0x7f003400b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 shutdown_connections 2024-01-19T04:55:45.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f002c06aed0 0x7f002c06d390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f004c0d3e50 0x7f004c0b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0d18e0 0x7f004c0b7d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 --2- 172.21.15.31:0/3317891428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f004c0d0ef0 0x7f004c1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 >> 172.21.15.31:0/3317891428 conn(0x7f004c09c600 msgr2=0x7f004c0afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:45.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 shutdown_connections 2024-01-19T04:55:45.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.677+0000 7f0050a2f640 1 -- 172.21.15.31:0/3317891428 wait complete. 2024-01-19T04:55:45.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:45.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.789+0000 7f5578cec640 1 Processor -- start 2024-01-19T04:55:45.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.789+0000 7f5578cec640 1 -- start start 2024-01-19T04:55:45.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5578cec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5574160bb0 0x7f55740b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5578cec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55740d1600 0x7f55740d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5578cec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55740d06b0 con 0x7f55740cfd10 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5578cec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55740b66b0 con 0x7f55740d1600 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5578cec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55740d1b90 con 0x7f5574160bb0 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5572d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55740d1600 0x7f55740d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f556bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f55740cfd10 0x7f55740d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53756/0 (socket says 172.21.15.31:53756) 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f556bfff640 1 -- 172.21.15.31:0/3248713371 learned_addr learned my addr 172.21.15.31:0/3248713371 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f557259c640 1 --2- 172.21.15.31:0/3248713371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5574160bb0 0x7f55740b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5572d9d640 1 -- 172.21.15.31:0/3248713371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5574160bb0 msgr2=0x7f55740b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5572d9d640 1 --2- 172.21.15.31:0/3248713371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5574160bb0 0x7f55740b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5572d9d640 1 -- 172.21.15.31:0/3248713371 >> v1:172.21.15.31:6789/0 conn(0x7f55740cfd10 legacy=0x7f55740d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5571d9b640 1 -- 172.21.15.31:0/3248713371 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651494004 0 0) 0x7f55740d06b0 con 0x7f55740cfd10 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.790+0000 7f5572d9d640 1 -- 172.21.15.31:0/3248713371 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55740d1e70 con 0x7f55740d1600 2024-01-19T04:55:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.791+0000 7f5572d9d640 1 --2- 172.21.15.31:0/3248713371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740d1600 0x7f55740d0170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f556c009d90 tx=0x7f556c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f02610fae5dad46 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.791+0000 7f5571d9b640 1 -- 172.21.15.31:0/3248713371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c02cc50 con 0x7f55740d1600 2024-01-19T04:55:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.791+0000 7f5571d9b640 1 -- 172.21.15.31:0/3248713371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c02cdb0 con 0x7f55740d1600 2024-01-19T04:55:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.791+0000 7f5571d9b640 1 -- 172.21.15.31:0/3248713371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c034660 con 0x7f55740d1600 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 -- 172.21.15.31:0/3248713371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740d1600 msgr2=0x7f55740d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 --2- 172.21.15.31:0/3248713371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740d1600 0x7f55740d0170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f556c009d90 tx=0x7f556c02bea0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 -- 172.21.15.31:0/3248713371 shutdown_connections 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 --2- 172.21.15.31:0/3248713371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740d1600 0x7f55740d0170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 --2- 172.21.15.31:0/3248713371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5574160bb0 0x7f55740b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 -- 172.21.15.31:0/3248713371 >> 172.21.15.31:0/3248713371 conn(0x7f557409c600 msgr2=0x7f55740d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 -- 172.21.15.31:0/3248713371 shutdown_connections 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.792+0000 7f5578cec640 1 -- 172.21.15.31:0/3248713371 wait complete. 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.793+0000 7f5578cec640 1 Processor -- start 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.793+0000 7f5578cec640 1 -- start start 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5578cec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740cfd10 0x7f55740c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5578cec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55740d1600 0x7f55740cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5578cec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5574160bb0 0x7f5574152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5578cec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740ca240 con 0x7f55740cfd10 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5578cec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740cb860 con 0x7f5574160bb0 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5578cec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55741f2c20 con 0x7f55740d1600 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740cfd10 0x7f55740c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740cfd10 0x7f55740c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53838/0 (socket says 172.21.15.31:53838) 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 -- 172.21.15.31:0/744110921 learned_addr learned my addr 172.21.15.31:0/744110921 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f556bfff640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55740d1600 0x7f55740cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5572d9d640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5574160bb0 0x7f5574152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 -- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55740d1600 msgr2=0x7f55740cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55740d1600 0x7f55740cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 -- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5574160bb0 msgr2=0x7f5574152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5574160bb0 0x7f5574152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 -- 172.21.15.31:0/744110921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c009a70 con 0x7f55740cfd10 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5572d9d640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5574160bb0 0x7f5574152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f557259c640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740cfd10 0x7f55740c9b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f556400d830 tx=0x7f556400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f556affd640 1 -- 172.21.15.31:0/744110921 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400b6c0 con 0x7f55740cfd10 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f556affd640 1 -- 172.21.15.31:0/744110921 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556400b820 con 0x7f55740cfd10 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.794+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55740caf30 con 0x7f55740cfd10 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.795+0000 7f556affd640 1 -- 172.21.15.31:0/744110921 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400bbb0 con 0x7f55740cfd10 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.795+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55740cb4f0 con 0x7f55740cfd10 2024-01-19T04:55:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.796+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557406ca10 con 0x7f55740cfd10 2024-01-19T04:55:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.796+0000 7f556affd640 1 -- 172.21.15.31:0/744110921 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f556401e460 con 0x7f55740cfd10 2024-01-19T04:55:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.796+0000 7f556affd640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f555406aed0 0x7f555406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.796+0000 7f556affd640 1 -- 172.21.15.31:0/744110921 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5564089d00 con 0x7f55740cfd10 2024-01-19T04:55:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.797+0000 7f556bfff640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f555406aed0 0x7f555406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.797+0000 7f556bfff640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f555406aed0 0x7f555406d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f55741f62b0 tx=0x7f555c007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.801+0000 7f556affd640 1 -- 172.21.15.31:0/744110921 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5564054bd0 con 0x7f55740cfd10 2024-01-19T04:55:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.982+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55740696e0 con 0x7f55740cfd10 2024-01-19T04:55:45.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.983+0000 7f556affd640 1 -- 172.21.15.31:0/744110921 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f55640581f0 con 0x7f55740cfd10 2024-01-19T04:55:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:45.983 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f555406aed0 msgr2=0x7f555406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f555406aed0 0x7f555406d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f55741f62b0 tx=0x7f555c007de0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740cfd10 msgr2=0x7f55740c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740cfd10 0x7f55740c9b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f556400d830 tx=0x7f556400dcf0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 shutdown_connections 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f555406aed0 0x7f555406d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5574160bb0 0x7f5574152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55740d1600 0x7f55740cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 --2- 172.21.15.31:0/744110921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55740cfd10 0x7f55740c9b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 >> 172.21.15.31:0/744110921 conn(0x7f557409c600 msgr2=0x7f557406c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 shutdown_connections 2024-01-19T04:55:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:45.986+0000 7f5578cec640 1 -- 172.21.15.31:0/744110921 wait complete. 2024-01-19T04:55:45.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:45.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.100+0000 7f7cb8301640 1 Processor -- start 2024-01-19T04:55:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.100+0000 7f7cb8301640 1 -- start start 2024-01-19T04:55:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.101+0000 7f7cb8301640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7cb006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.101+0000 7f7cb8301640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb00cf9f0 con 0x7f7cb0161860 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.101+0000 7f7cb8301640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb00cfcd0 con 0x7f7cb00cf570 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.101+0000 7f7cb8301640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00cffb0 con 0x7f7cb015d3a0 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.101+0000 7f7cb689e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7cb0161860 0x7f7cb0167480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53762/0 (socket says 172.21.15.31:53762) 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.101+0000 7f7cb689e640 1 -- 172.21.15.31:0/3885076469 learned_addr learned my addr 172.21.15.31:0/3885076469 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb609d640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7cb006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760317267 0 0) 0x7f7cb00cf9f0 con 0x7f7cb0161860 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c9c003520 con 0x7f7cb0161860 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709114691 0 0) 0x7f7cb00cfcd0 con 0x7f7cb00cf570 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb00cf9f0 con 0x7f7cb00cf570 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164934938 0 0) 0x7f7c9c003520 con 0x7f7cb0161860 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 msgr2=0x7f7cb006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7cb006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 >> v1:172.21.15.137:6789/0 conn(0x7f7cb00cf570 legacy=0x7f7cb0161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00d0290 con 0x7f7cb0161860 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f7ca80019f0 con 0x7f7cb0161860 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb609d640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7cb006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 >> v1:172.21.15.31:6789/0 conn(0x7f7cb0161860 legacy=0x7f7cb0167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7c9c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c9c004070 0x7f7c9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c9c004d00 0x7f7c9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00cffb0 con 0x7f7c9c004d00 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c00ba30 con 0x7f7c9c004070 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c00bbf0 con 0x7f7cb015d3a0 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb689e640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c9c004d00 0x7f7c9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb609d640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7c9c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb589c640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c9c004070 0x7f7c9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb689e640 1 -- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 msgr2=0x7f7c9c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb689e640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7c9c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb689e640 1 -- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c9c004070 msgr2=0x7f7c9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb689e640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c9c004070 0x7f7c9c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb689e640 1 -- 172.21.15.31:0/3885076469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0093a0 con 0x7f7c9c004d00 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.102+0000 7f7cb609d640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cb015d3a0 0x7f7c9c003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.103+0000 7f7cb689e640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c9c004d00 0x7f7c9c005180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800a7c0 tx=0x7f7c9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e972595efecd340e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.103+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8034630 con 0x7f7c9c004d00 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.103+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca802f070 con 0x7f7c9c004d00 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.103+0000 7f7cb509b640 1 -- 172.21.15.31:0/3885076469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80349f0 con 0x7f7c9c004d00 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.104+0000 7f7cb8301640 1 -- 172.21.15.31:0/3885076469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c9c004d00 msgr2=0x7f7c9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.104+0000 7f7cb8301640 1 --2- 172.21.15.31:0/3885076469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c9c004d00 0x7f7c9c005180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800a7c0 tx=0x7f7c9c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.104+0000 7f7cb689e640 1 -- 172.21.15.31:0/3885076469 reap_dead start 2024-01-19T04:55:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.104+0000 7f7cb8301640 1 -- 172.21.15.31:0/3885076469 shutdown_connections 2024-01-19T04:55:46.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.104+0000 7f7cb8301640 1 -- 172.21.15.31:0/3885076469 >> 172.21.15.31:0/3885076469 conn(0x7f7cb00d1100 msgr2=0x7f7cb00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:46.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.104+0000 7f7cb8301640 1 -- 172.21.15.31:0/3885076469 shutdown_connections 2024-01-19T04:55:46.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.104+0000 7f7cb8301640 1 -- 172.21.15.31:0/3885076469 wait complete. 2024-01-19T04:55:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.105+0000 7f7cb8301640 1 Processor -- start 2024-01-19T04:55:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.105+0000 7f7cb8301640 1 -- start start 2024-01-19T04:55:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c9c004d00 0x7f7cb00c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb014a4a0 0x7f7cb01e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cb01e8910 0x7f7cb01f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb01e8d90 con 0x7f7cb014a4a0 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c7670 con 0x7f7cb01e8910 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c6ca0 con 0x7f7c9c004d00 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb014a4a0 0x7f7cb01e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb014a4a0 0x7f7cb01e83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53848/0 (socket says 172.21.15.31:53848) 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 -- 172.21.15.31:0/1425318152 learned_addr learned my addr 172.21.15.31:0/1425318152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 -- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c9c004d00 msgr2=0x7f7cb00c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c9c004d00 0x7f7cb00c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 -- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cb01e8910 msgr2=0x7f7cb01f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cb01e8910 0x7f7cb01f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 -- 172.21.15.31:0/1425318152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb01f7840 con 0x7f7cb014a4a0 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb589c640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb014a4a0 0x7f7cb01e83d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000f800 tx=0x7f7ca000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7ca6ffd640 1 -- 172.21.15.31:0/1425318152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000feb0 con 0x7f7cb014a4a0 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7ca6ffd640 1 -- 172.21.15.31:0/1425318152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0005320 con 0x7f7cb014a4a0 2024-01-19T04:55:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb00c6370 con 0x7f7cb014a4a0 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7ca6ffd640 1 -- 172.21.15.31:0/1425318152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0007290 con 0x7f7cb014a4a0 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.106+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb00c6930 con 0x7f7cb014a4a0 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.107+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb0074900 con 0x7f7cb014a4a0 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.108+0000 7f7ca6ffd640 1 -- 172.21.15.31:0/1425318152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7ca00073f0 con 0x7f7cb014a4a0 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.108+0000 7f7ca6ffd640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.108+0000 7f7ca6ffd640 1 -- 172.21.15.31:0/1425318152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ca0088fd0 con 0x7f7cb014a4a0 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.109+0000 7f7cb609d640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.109+0000 7f7cb609d640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c9406aed0 0x7f7c9406d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00b3c50 tx=0x7f7cac008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.113+0000 7f7ca6ffd640 1 -- 172.21.15.31:0/1425318152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7ca0053f60 con 0x7f7cb014a4a0 2024-01-19T04:55:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.293+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cb0067030 con 0x7f7cb014a4a0 2024-01-19T04:55:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.293+0000 7f7ca6ffd640 1 -- 172.21.15.31:0/1425318152 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ca0005490 con 0x7f7cb014a4a0 2024-01-19T04:55:46.293 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.295+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c9406aed0 msgr2=0x7f7c9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.295+0000 7f7cb8301640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c9406aed0 0x7f7c9406d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00b3c50 tx=0x7f7cac008040 comp rx=0 tx=0).stop 2024-01-19T04:55:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.295+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb014a4a0 msgr2=0x7f7cb01e83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.295+0000 7f7cb8301640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb014a4a0 0x7f7cb01e83d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000f800 tx=0x7f7ca000fcc0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.295+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 shutdown_connections 2024-01-19T04:55:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.296+0000 7f7cb8301640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.296+0000 7f7cb8301640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cb01e8910 0x7f7cb01f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.296+0000 7f7cb8301640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb014a4a0 0x7f7cb01e83d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.296+0000 7f7cb8301640 1 --2- 172.21.15.31:0/1425318152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c9c004d00 0x7f7cb00c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.296+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 >> 172.21.15.31:0/1425318152 conn(0x7f7cb00d1100 msgr2=0x7f7cb015dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.296+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 shutdown_connections 2024-01-19T04:55:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.296+0000 7f7cb8301640 1 -- 172.21.15.31:0/1425318152 wait complete. 2024-01-19T04:55:46.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.409+0000 7f439230e640 1 Processor -- start 2024-01-19T04:55:46.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.409+0000 7f439230e640 1 -- start start 2024-01-19T04:55:46.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.410+0000 7f439230e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d3e50 0x7f438c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.410+0000 7f439230e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f438c0d0ef0 0x7f438c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.410+0000 7f439230e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438c15dee0 con 0x7f438c0d18e0 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.410+0000 7f439230e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438c15e1c0 con 0x7f438c0d0ef0 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.410+0000 7f439230e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c15e4a0 con 0x7f438c0d3e50 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.410+0000 7f43908ab640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f438c0d0ef0 0x7f438c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.410+0000 7f438b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d3e50 0x7f438c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f438bfff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f438c0d18e0 0x7f438c0d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:60742/0 (socket says 172.21.15.31:60742) 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f438bfff640 1 -- 172.21.15.31:0/455426628 learned_addr learned my addr 172.21.15.31:0/455426628 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f43908ab640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f438c0d0ef0 0x7f438c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35824/0 (socket says 172.21.15.31:35824) 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f438b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d3e50 0x7f438c15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:57034/0 (socket says 172.21.15.31:57034) 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f43908ab640 1 -- 172.21.15.31:0/455426628 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d3e50 msgr2=0x7f438c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f43908ab640 1 --2- 172.21.15.31:0/455426628 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d3e50 0x7f438c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f43908ab640 1 -- 172.21.15.31:0/455426628 >> v1:172.21.15.137:6789/0 conn(0x7f438c0d18e0 legacy=0x7f438c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f438affd640 1 -- 172.21.15.31:0/455426628 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772458033 0 0) 0x7f438c15dee0 con 0x7f438c0d18e0 2024-01-19T04:55:46.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.411+0000 7f43908ab640 1 -- 172.21.15.31:0/455426628 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c15e780 con 0x7f438c0d0ef0 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f43908ab640 1 --2- 172.21.15.31:0/455426628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d0ef0 0x7f438c0cdfe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4384009d90 tx=0x7f438402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f60c79a1841c93 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f438affd640 1 -- 172.21.15.31:0/455426628 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438402cc50 con 0x7f438c0d0ef0 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f438affd640 1 -- 172.21.15.31:0/455426628 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438402cdb0 con 0x7f438c0d0ef0 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f438affd640 1 -- 172.21.15.31:0/455426628 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384034660 con 0x7f438c0d0ef0 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 -- 172.21.15.31:0/455426628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d0ef0 msgr2=0x7f438c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 --2- 172.21.15.31:0/455426628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d0ef0 0x7f438c0cdfe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4384009d90 tx=0x7f438402bea0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 -- 172.21.15.31:0/455426628 shutdown_connections 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 --2- 172.21.15.31:0/455426628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d0ef0 0x7f438c0cdfe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 --2- 172.21.15.31:0/455426628 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d3e50 0x7f438c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 -- 172.21.15.31:0/455426628 >> 172.21.15.31:0/455426628 conn(0x7f438c09c600 msgr2=0x7f438c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:46.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 -- 172.21.15.31:0/455426628 shutdown_connections 2024-01-19T04:55:46.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.412+0000 7f439230e640 1 -- 172.21.15.31:0/455426628 wait complete. 2024-01-19T04:55:46.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 Processor -- start 2024-01-19T04:55:46.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 -- start start 2024-01-19T04:55:46.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d0ef0 0x7f438c06c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f438c0d18e0 0x7f438c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d3e50 0x7f438c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0ca3f0 con 0x7f438c0d18e0 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0c8ae0 con 0x7f438c0d3e50 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.414+0000 7f439230e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0c8dc0 con 0x7f438c0d0ef0 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f438c0d18e0 0x7f438c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f438c0d18e0 0x7f438c153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53850/0 (socket says 172.21.15.31:53850) 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 -- 172.21.15.31:0/248737239 learned_addr learned my addr 172.21.15.31:0/248737239 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f43908ab640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d3e50 0x7f438c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438bfff640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d0ef0 0x7f438c06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 -- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d0ef0 msgr2=0x7f438c06c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d0ef0 0x7f438c06c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 -- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d3e50 msgr2=0x7f438c0cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d3e50 0x7f438c0cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 -- 172.21.15.31:0/248737239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384009a70 con 0x7f438c0d18e0 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f438b7fe640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f438c0d18e0 0x7f438c153750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f437400e870 tx=0x7f437400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f4388ff9640 1 -- 172.21.15.31:0/248737239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43740094f0 con 0x7f438c0d18e0 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f4388ff9640 1 -- 172.21.15.31:0/248737239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437400c940 con 0x7f438c0d18e0 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c0ca640 con 0x7f438c0d18e0 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f4388ff9640 1 -- 172.21.15.31:0/248737239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437400ccf0 con 0x7f438c0d18e0 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.415+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c06cdc0 con 0x7f438c0d18e0 2024-01-19T04:55:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.416+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c074900 con 0x7f438c0d18e0 2024-01-19T04:55:46.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.417+0000 7f4388ff9640 1 -- 172.21.15.31:0/248737239 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4374013470 con 0x7f438c0d18e0 2024-01-19T04:55:46.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.417+0000 7f4388ff9640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f436c06aed0 0x7f436c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.417+0000 7f4388ff9640 1 -- 172.21.15.31:0/248737239 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4374089660 con 0x7f438c0d18e0 2024-01-19T04:55:46.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.417+0000 7f438bfff640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f436c06aed0 0x7f436c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.418+0000 7f438bfff640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f436c06aed0 0x7f436c06d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f437c000f60 tx=0x7f437c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.422+0000 7f4388ff9640 1 -- 172.21.15.31:0/248737239 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f43740545f0 con 0x7f438c0d18e0 2024-01-19T04:55:46.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.600+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f438c0d09a0 con 0x7f438c0d18e0 2024-01-19T04:55:46.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.600+0000 7f4388ff9640 1 -- 172.21.15.31:0/248737239 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4374057c10 con 0x7f438c0d18e0 2024-01-19T04:55:46.600 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:46.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f436c06aed0 msgr2=0x7f436c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f436c06aed0 0x7f436c06d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f437c000f60 tx=0x7f437c007d50 comp rx=0 tx=0).stop 2024-01-19T04:55:46.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f438c0d18e0 msgr2=0x7f438c153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f438c0d18e0 0x7f438c153750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f437400e870 tx=0x7f437400ed30 comp rx=0 tx=0).stop 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 shutdown_connections 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f436c06aed0 0x7f436c06d390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438c0d3e50 0x7f438c0cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f438c0d18e0 0x7f438c153750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 --2- 172.21.15.31:0/248737239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438c0d0ef0 0x7f438c06c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 >> 172.21.15.31:0/248737239 conn(0x7f438c09c600 msgr2=0x7f438c0d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 shutdown_connections 2024-01-19T04:55:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.603+0000 7f439230e640 1 -- 172.21.15.31:0/248737239 wait complete. 2024-01-19T04:55:46.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.717+0000 7fa578bde640 1 Processor -- start 2024-01-19T04:55:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.717+0000 7fa578bde640 1 -- start start 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa578bde640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5740d1c20 0x7fa5740d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa578bde640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa5740d3ea0 0x7fa5740b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa578bde640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740cf160 con 0x7fa57415d720 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa578bde640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740ce8c0 con 0x7fa5740d1c20 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa578bde640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740ceba0 con 0x7fa5740d3ea0 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa571d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5740d1c20 0x7fa5740d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa57259c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa57415d720 0x7fa5740d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53776/0 (socket says 172.21.15.31:53776) 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.718+0000 7fa57259c640 1 -- 172.21.15.31:0/664903868 learned_addr learned my addr 172.21.15.31:0/664903868 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa571d9b640 1 -- 172.21.15.31:0/664903868 >> v2:172.21.15.137:3300/0 conn(0x7fa5740d3ea0 msgr2=0x7fa5740b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa571d9b640 1 --2- 172.21.15.31:0/664903868 >> v2:172.21.15.137:3300/0 conn(0x7fa5740d3ea0 0x7fa5740b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa571d9b640 1 -- 172.21.15.31:0/664903868 >> v1:172.21.15.31:6789/0 conn(0x7fa57415d720 legacy=0x7fa5740d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa571d9b640 1 -- 172.21.15.31:0/664903868 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740cee80 con 0x7fa5740d1c20 2024-01-19T04:55:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa57159a640 1 -- 172.21.15.31:0/664903868 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790070285 0 0) 0x7fa5740cf160 con 0x7fa57415d720 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa571d9b640 1 --2- 172.21.15.31:0/664903868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740d1c20 0x7fa5740d0ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa5741611a0 tx=0x7fa56802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5746ad4de51e62ab server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa57159a640 1 -- 172.21.15.31:0/664903868 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568031710 con 0x7fa5740d1c20 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa57159a640 1 -- 172.21.15.31:0/664903868 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56802f050 con 0x7fa5740d1c20 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.719+0000 7fa57159a640 1 -- 172.21.15.31:0/664903868 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568031a90 con 0x7fa5740d1c20 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 -- 172.21.15.31:0/664903868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740d1c20 msgr2=0x7fa5740d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 --2- 172.21.15.31:0/664903868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740d1c20 0x7fa5740d0ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa5741611a0 tx=0x7fa56802b7b0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 -- 172.21.15.31:0/664903868 shutdown_connections 2024-01-19T04:55:46.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 --2- 172.21.15.31:0/664903868 >> v2:172.21.15.137:3300/0 conn(0x7fa5740d3ea0 0x7fa5740b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 --2- 172.21.15.31:0/664903868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740d1c20 0x7fa5740d0ee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 -- 172.21.15.31:0/664903868 >> 172.21.15.31:0/664903868 conn(0x7fa5740d3310 msgr2=0x7fa5740d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 -- 172.21.15.31:0/664903868 shutdown_connections 2024-01-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.720+0000 7fa578bde640 1 -- 172.21.15.31:0/664903868 wait complete. 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.721+0000 7fa578bde640 1 Processor -- start 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.721+0000 7fa578bde640 1 -- start start 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa578bde640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5740d1c20 0x7fa5740ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa578bde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5740d3ea0 0x7fa5740d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa578bde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57415d720 0x7fa5741597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa578bde640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740ca920 con 0x7fa57415d720 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa578bde640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740d0c20 con 0x7fa5740d3ea0 2024-01-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa578bde640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c7450 con 0x7fa5740d1c20 2024-01-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa572d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57415d720 0x7fa5741597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa572d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57415d720 0x7fa5741597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53868/0 (socket says 172.21.15.31:53868) 2024-01-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa572d9d640 1 -- 172.21.15.31:0/3930731170 learned_addr learned my addr 172.21.15.31:0/3930731170 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa572d9d640 1 -- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5740d1c20 msgr2=0x7fa5740ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa572d9d640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5740d1c20 0x7fa5740ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa572d9d640 1 -- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5740d3ea0 msgr2=0x7fa5740d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.722+0000 7fa571d9b640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5740d3ea0 0x7fa5740d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa572d9d640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5740d3ea0 0x7fa5740d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa572d9d640 1 -- 172.21.15.31:0/3930731170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568009400 con 0x7fa57415d720 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa571d9b640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5740d3ea0 0x7fa5740d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa572d9d640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57415d720 0x7fa5741597f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009fd0 tx=0x7fa56c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa562ffd640 1 -- 172.21.15.31:0/3930731170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c011830 con 0x7fa57415d720 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa562ffd640 1 -- 172.21.15.31:0/3930731170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c011990 con 0x7fa57415d720 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa562ffd640 1 -- 172.21.15.31:0/3930731170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c011d20 con 0x7fa57415d720 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5740c7730 con 0x7fa57415d720 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.723+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57415ee30 con 0x7fa57415d720 2024-01-19T04:55:46.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.724+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5740c7c50 con 0x7fa57415d720 2024-01-19T04:55:46.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.724+0000 7fa562ffd640 1 -- 172.21.15.31:0/3930731170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa56c01b460 con 0x7fa57415d720 2024-01-19T04:55:46.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.725+0000 7fa562ffd640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa55406aed0 0x7fa55406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:46.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.725+0000 7fa562ffd640 1 -- 172.21.15.31:0/3930731170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa56c089440 con 0x7fa57415d720 2024-01-19T04:55:46.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.725+0000 7fa57259c640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa55406aed0 0x7fa55406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:46.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.726+0000 7fa57259c640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa55406aed0 0x7fa55406d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564007ce0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:46.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.729+0000 7fa562ffd640 1 -- 172.21.15.31:0/3930731170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa5740c7c50 con 0x7fa57415d720 2024-01-19T04:55:46.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.905+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5740c7c50 con 0x7fa57415d720 2024-01-19T04:55:46.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.905+0000 7fa562ffd640 1 -- 172.21.15.31:0/3930731170 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5740c7c50 con 0x7fa57415d720 2024-01-19T04:55:46.905 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:46.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa55406aed0 msgr2=0x7fa55406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa55406aed0 0x7fa55406d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564007ce0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57415d720 msgr2=0x7fa5741597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:46.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57415d720 0x7fa5741597f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009fd0 tx=0x7fa56c00b8e0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 shutdown_connections 2024-01-19T04:55:46.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa55406aed0 0x7fa55406d390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57415d720 0x7fa5741597f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5740d3ea0 0x7fa5740d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 --2- 172.21.15.31:0/3930731170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5740d1c20 0x7fa5740ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:46.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 >> 172.21.15.31:0/3930731170 conn(0x7fa5740d3310 msgr2=0x7fa5740b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:46.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 shutdown_connections 2024-01-19T04:55:46.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:46.907+0000 7fa578bde640 1 -- 172.21.15.31:0/3930731170 wait complete. 2024-01-19T04:55:46.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.018+0000 7f80c177c640 1 Processor -- start 2024-01-19T04:55:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.018+0000 7f80c177c640 1 -- start start 2024-01-19T04:55:47.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80c177c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 0x7f80bc0d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80c177c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f80bc0cee00 0x7f80bc15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80c177c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80bc0d0040 con 0x7f80bc06ecb0 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80c177c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80bc15c800 con 0x7f80bc0cee00 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80c177c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc0cd500 con 0x7f80bc06e8d0 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80bb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f80bc06ecb0 0x7f80bc0cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53784/0 (socket says 172.21.15.31:53784) 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80bb7fe640 1 -- 172.21.15.31:0/4277431607 learned_addr learned my addr 172.21.15.31:0/4277431607 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80baffd640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 0x7f80bc0d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.019+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/4277431607 >> v2:172.21.15.137:3300/0 conn(0x7f80bc0cee00 0x7f80bc15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993863914 0 0) 0x7f80bc0d0040 con 0x7f80bc06ecb0 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a4003520 con 0x7f80bc06ecb0 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1448767198 0 0) 0x7f80a4003520 con 0x7f80bc06ecb0 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 msgr2=0x7f80bc0d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 0x7f80bc0d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 >> v2:172.21.15.137:3300/0 conn(0x7f80bc0cee00 msgr2=0x7f80bc15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 --2- 172.21.15.31:0/4277431607 >> v2:172.21.15.137:3300/0 conn(0x7f80bc0cee00 0x7f80bc15c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc0cb800 con 0x7f80bc06ecb0 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f80ac0019f0 con 0x7f80bc06ecb0 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80baffd640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 0x7f80bc0d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 >> v1:172.21.15.31:6789/0 conn(0x7f80bc06ecb0 legacy=0x7f80bc0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 0x7f80a4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4003eb0 0x7f80a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4004d00 0x7f80a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc15c800 con 0x7f80a4003eb0 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc0cd500 con 0x7f80a4004d00 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a400b980 con 0x7f80bc06e8d0 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/4277431607 >> v2:172.21.15.137:3300/0 conn(0x7f80bc0cee00 0x7f80bc15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4003eb0 0x7f80a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80bb7fe640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4004d00 0x7f80a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80baffd640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 0x7f80a4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80ba7fc640 1 -- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 msgr2=0x7f80a4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc06e8d0 0x7f80a4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80ba7fc640 1 -- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4004d00 msgr2=0x7f80a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4004d00 0x7f80a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.020+0000 7f80ba7fc640 1 -- 172.21.15.31:0/4277431607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4009400 con 0x7f80a4003eb0 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.021+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4003eb0 0x7f80a40047c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009c70 tx=0x7f80a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9c9baa52edf0df server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.021+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0004020 con 0x7f80a4003eb0 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.021+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b0004180 con 0x7f80a4003eb0 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.021+0000 7f80b9ffb640 1 -- 172.21.15.31:0/4277431607 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b002c960 con 0x7f80a4003eb0 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.022+0000 7f80c177c640 1 -- 172.21.15.31:0/4277431607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4003eb0 msgr2=0x7f80a40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.022+0000 7f80c177c640 1 --2- 172.21.15.31:0/4277431607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4003eb0 0x7f80a40047c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009c70 tx=0x7f80a40057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.022+0000 7f80bb7fe640 1 -- 172.21.15.31:0/4277431607 reap_dead start 2024-01-19T04:55:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.022+0000 7f80c177c640 1 -- 172.21.15.31:0/4277431607 shutdown_connections 2024-01-19T04:55:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.022+0000 7f80c177c640 1 -- 172.21.15.31:0/4277431607 >> 172.21.15.31:0/4277431607 conn(0x7f80bc0b2c60 msgr2=0x7f80bc0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.022+0000 7f80c177c640 1 -- 172.21.15.31:0/4277431607 shutdown_connections 2024-01-19T04:55:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.022+0000 7f80c177c640 1 -- 172.21.15.31:0/4277431607 wait complete. 2024-01-19T04:55:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.023+0000 7f80c177c640 1 Processor -- start 2024-01-19T04:55:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.023+0000 7f80c177c640 1 -- start start 2024-01-19T04:55:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80c177c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4003eb0 0x7f80bc15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80c177c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80bc15bbb0 0x7f80bc159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80c177c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc1598a0 0x7f80bc0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80c177c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc159d50 con 0x7f80bc15bbb0 2024-01-19T04:55:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80c177c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc15b1e0 con 0x7f80a4003eb0 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80c177c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc0b6a70 con 0x7f80bc1598a0 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80bc15bbb0 0x7f80bc159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80bc15bbb0 0x7f80bc159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53892/0 (socket says 172.21.15.31:53892) 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 -- 172.21.15.31:0/660647110 learned_addr learned my addr 172.21.15.31:0/660647110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80bb7fe640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc1598a0 0x7f80bc0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80baffd640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4003eb0 0x7f80bc15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 -- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc1598a0 msgr2=0x7f80bc0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc1598a0 0x7f80bc0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 -- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4003eb0 msgr2=0x7f80bc15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4003eb0 0x7f80bc15b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 -- 172.21.15.31:0/660647110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc1f5090 con 0x7f80bc15bbb0 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80baffd640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4003eb0 0x7f80bc15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.024+0000 7f80ba7fc640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80bc15bbb0 0x7f80bc159360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009d60 tx=0x7f80b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.025+0000 7f80a37fe640 1 -- 172.21.15.31:0/660647110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0005e90 con 0x7f80bc15bbb0 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.025+0000 7f80a37fe640 1 -- 172.21.15.31:0/660647110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b002cb70 con 0x7f80bc15bbb0 2024-01-19T04:55:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.025+0000 7f80a37fe640 1 -- 172.21.15.31:0/660647110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0006040 con 0x7f80bc15bbb0 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.025+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80bc1f4db0 con 0x7f80bc15bbb0 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.025+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80bc15ae70 con 0x7f80bc15bbb0 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.026+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80bc074a80 con 0x7f80bc15bbb0 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.026+0000 7f80a37fe640 1 -- 172.21.15.31:0/660647110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f80b00061a0 con 0x7f80bc15bbb0 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.026+0000 7f80a37fe640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a806af00 0x7f80a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.026+0000 7f80a37fe640 1 -- 172.21.15.31:0/660647110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80b00abbe0 con 0x7f80bc15bbb0 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.027+0000 7f80baffd640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a806af00 0x7f80a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.027+0000 7f80baffd640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a806af00 0x7f80a806d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009b70 tx=0x7f80b4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.031+0000 7f80a37fe640 1 -- 172.21.15.31:0/660647110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f80b0076b40 con 0x7f80bc15bbb0 2024-01-19T04:55:47.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.208+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80bc15a9f0 con 0x7f80bc15bbb0 2024-01-19T04:55:47.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.210+0000 7f80a37fe640 1 -- 172.21.15.31:0/660647110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f80bc15a9f0 con 0x7f80bc15bbb0 2024-01-19T04:55:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:47.209 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.211+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a806af00 msgr2=0x7f80a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.211+0000 7f80c177c640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a806af00 0x7f80a806d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009b70 tx=0x7f80b4008040 comp rx=0 tx=0).stop 2024-01-19T04:55:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.211+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80bc15bbb0 msgr2=0x7f80bc159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.211+0000 7f80c177c640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80bc15bbb0 0x7f80bc159360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009d60 tx=0x7f80b0004730 comp rx=0 tx=0).stop 2024-01-19T04:55:47.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 shutdown_connections 2024-01-19T04:55:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a806af00 0x7f80a806d3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80bc1598a0 0x7f80bc0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80bc15bbb0 0x7f80bc159360 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 --2- 172.21.15.31:0/660647110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a4003eb0 0x7f80bc15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 >> 172.21.15.31:0/660647110 conn(0x7f80bc0b2c60 msgr2=0x7f80bc15df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 shutdown_connections 2024-01-19T04:55:47.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.212+0000 7f80c177c640 1 -- 172.21.15.31:0/660647110 wait complete. 2024-01-19T04:55:47.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:47.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.327+0000 7f363119b640 1 Processor -- start 2024-01-19T04:55:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.327+0000 7f363119b640 1 -- start start 2024-01-19T04:55:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f363119b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f362c0cf570 0x7f362c161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f363119b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d780 0x7f362c161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f363119b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0cf9f0 con 0x7f362c15d3a0 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f363119b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0cfcd0 con 0x7f362c0cf570 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f363119b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0cffb0 con 0x7f362c15d780 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f362c0cf570 0x7f362c161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f362c15d3a0 0x7f362c06c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53790/0 (socket says 172.21.15.31:53790) 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362ad9d640 1 -- 172.21.15.31:0/3273522791 learned_addr learned my addr 172.21.15.31:0/3273522791 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362a59c640 1 -- 172.21.15.31:0/3273522791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d780 msgr2=0x7f362c161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362b59e640 1 --2- 172.21.15.31:0/3273522791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d780 0x7f362c161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f3629d9b640 1 -- 172.21.15.31:0/3273522791 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335399632 0 0) 0x7f362c0cf9f0 con 0x7f362c15d3a0 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362a59c640 1 --2- 172.21.15.31:0/3273522791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d780 0x7f362c161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362a59c640 1 -- 172.21.15.31:0/3273522791 >> v1:172.21.15.31:6789/0 conn(0x7f362c15d3a0 legacy=0x7f362c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362a59c640 1 -- 172.21.15.31:0/3273522791 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0d0290 con 0x7f362c0cf570 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.328+0000 7f362b59e640 1 --2- 172.21.15.31:0/3273522791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d780 0x7f362c161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.329+0000 7f362a59c640 1 --2- 172.21.15.31:0/3273522791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c161150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36140097a0 tx=0x7f361402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe504c1a9a2ba7ce server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.329+0000 7f3629d9b640 1 -- 172.21.15.31:0/3273522791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614031830 con 0x7f362c0cf570 2024-01-19T04:55:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.329+0000 7f3629d9b640 1 -- 172.21.15.31:0/3273522791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3614031990 con 0x7f362c0cf570 2024-01-19T04:55:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.329+0000 7f3629d9b640 1 -- 172.21.15.31:0/3273522791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614031d20 con 0x7f362c0cf570 2024-01-19T04:55:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 -- 172.21.15.31:0/3273522791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 msgr2=0x7f362c161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 --2- 172.21.15.31:0/3273522791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c161150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36140097a0 tx=0x7f361402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 -- 172.21.15.31:0/3273522791 shutdown_connections 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 --2- 172.21.15.31:0/3273522791 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d780 0x7f362c161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 --2- 172.21.15.31:0/3273522791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c161150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 -- 172.21.15.31:0/3273522791 >> 172.21.15.31:0/3273522791 conn(0x7f362c0d1100 msgr2=0x7f362c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 -- 172.21.15.31:0/3273522791 shutdown_connections 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.330+0000 7f363119b640 1 -- 172.21.15.31:0/3273522791 wait complete. 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.331+0000 7f363119b640 1 Processor -- start 2024-01-19T04:55:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.331+0000 7f363119b640 1 -- start start 2024-01-19T04:55:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f363119b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c1554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f363119b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d3a0 0x7f362c0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f363119b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f362c15d780 0x7f362c0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f363119b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0c5ee0 con 0x7f362c0cf570 2024-01-19T04:55:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f363119b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0c4b60 con 0x7f362c15d780 2024-01-19T04:55:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f363119b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0c4e40 con 0x7f362c15d3a0 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c1554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c1554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53910/0 (socket says 172.21.15.31:53910) 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 -- 172.21.15.31:0/2542444821 learned_addr learned my addr 172.21.15.31:0/2542444821 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 -- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d3a0 msgr2=0x7f362c0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362a59c640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d3a0 0x7f362c0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362b59e640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f362c15d780 0x7f362c0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d3a0 0x7f362c0c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 -- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f362c15d780 msgr2=0x7f362c0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f362c15d780 0x7f362c0c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.332+0000 7f362ad9d640 1 -- 172.21.15.31:0/2542444821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009400 con 0x7f362c0cf570 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.333+0000 7f362ad9d640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c1554b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c840 tx=0x7f361c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.333+0000 7f36237fe640 1 -- 172.21.15.31:0/2542444821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00a640 con 0x7f362c0cf570 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.333+0000 7f36237fe640 1 -- 172.21.15.31:0/2542444821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00a7a0 con 0x7f362c0cf570 2024-01-19T04:55:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.333+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c0c5120 con 0x7f362c0cf570 2024-01-19T04:55:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.333+0000 7f36237fe640 1 -- 172.21.15.31:0/2542444821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00ab50 con 0x7f362c0cf570 2024-01-19T04:55:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.333+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c0c56e0 con 0x7f362c0cf570 2024-01-19T04:55:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.334+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c074900 con 0x7f362c0cf570 2024-01-19T04:55:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.334+0000 7f36237fe640 1 -- 172.21.15.31:0/2542444821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f361c00acb0 con 0x7f362c0cf570 2024-01-19T04:55:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.335+0000 7f36237fe640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f360c06aed0 0x7f360c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.335+0000 7f36237fe640 1 -- 172.21.15.31:0/2542444821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f361c0894e0 con 0x7f362c0cf570 2024-01-19T04:55:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.335+0000 7f362a59c640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f360c06aed0 0x7f360c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.335+0000 7f362a59c640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f360c06aed0 0x7f360c06d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f3614008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.339+0000 7f36237fe640 1 -- 172.21.15.31:0/2542444821 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f361c0543b0 con 0x7f362c0cf570 2024-01-19T04:55:47.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.517+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f362c067030 con 0x7f362c0cf570 2024-01-19T04:55:47.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.517+0000 7f36237fe640 1 -- 172.21.15.31:0/2542444821 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f361c0579d0 con 0x7f362c0cf570 2024-01-19T04:55:47.517 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:47.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f360c06aed0 msgr2=0x7f360c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f360c06aed0 0x7f360c06d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f3614008030 comp rx=0 tx=0).stop 2024-01-19T04:55:47.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 msgr2=0x7f362c1554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c1554b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c840 tx=0x7f361c00cd00 comp rx=0 tx=0).stop 2024-01-19T04:55:47.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 shutdown_connections 2024-01-19T04:55:47.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f360c06aed0 0x7f360c06d390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f362c15d780 0x7f362c0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f362c15d3a0 0x7f362c0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 --2- 172.21.15.31:0/2542444821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cf570 0x7f362c1554b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 >> 172.21.15.31:0/2542444821 conn(0x7f362c0d1100 msgr2=0x7f362c15dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:47.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 shutdown_connections 2024-01-19T04:55:47.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.519+0000 7f363119b640 1 -- 172.21.15.31:0/2542444821 wait complete. 2024-01-19T04:55:47.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:47.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.634+0000 7f40c862b640 1 Processor -- start 2024-01-19T04:55:47.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.634+0000 7f40c862b640 1 -- start start 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.634+0000 7f40c862b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f40c015d700 0x7f40c00d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.634+0000 7f40c862b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40c00d1fe0 0x7f40c00b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c862b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c00cef10 con 0x7f40c00d11b0 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c862b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00ce300 con 0x7f40c00d1fe0 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c862b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00ce5e0 con 0x7f40c015d700 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c6bc8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40c00d1fe0 0x7f40c00b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c6bc8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40c00d1fe0 0x7f40c00b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53924/0 (socket says 172.21.15.31:53924) 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c6bc8640 1 -- 172.21.15.31:0/1701321909 learned_addr learned my addr 172.21.15.31:0/1701321909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:47.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c63c7640 1 --2- 172.21.15.31:0/1701321909 >> v2:172.21.15.137:3300/0 conn(0x7f40c015d700 0x7f40c00d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c6bc8640 1 -- 172.21.15.31:0/1701321909 >> v2:172.21.15.137:3300/0 conn(0x7f40c015d700 msgr2=0x7f40c00d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c6bc8640 1 --2- 172.21.15.31:0/1701321909 >> v2:172.21.15.137:3300/0 conn(0x7f40c015d700 0x7f40c00d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c6bc8640 1 -- 172.21.15.31:0/1701321909 >> v1:172.21.15.137:6789/0 conn(0x7f40c00d11b0 legacy=0x7f40c00d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.635+0000 7f40c6bc8640 1 -- 172.21.15.31:0/1701321909 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00ce8c0 con 0x7f40c00d1fe0 2024-01-19T04:55:47.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c6bc8640 1 --2- 172.21.15.31:0/1701321909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c00d1fe0 0x7f40c00b2890 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40c00b5e70 tx=0x7f40b802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad90e6855200e16 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c53c5640 1 -- 172.21.15.31:0/1701321909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b802cb20 con 0x7f40c00d1fe0 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c53c5640 1 -- 172.21.15.31:0/1701321909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b802cc80 con 0x7f40c00d1fe0 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c53c5640 1 -- 172.21.15.31:0/1701321909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b80343f0 con 0x7f40c00d1fe0 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c862b640 1 -- 172.21.15.31:0/1701321909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c00d1fe0 msgr2=0x7f40c00b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c862b640 1 --2- 172.21.15.31:0/1701321909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c00d1fe0 0x7f40c00b2890 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40c00b5e70 tx=0x7f40b802bf80 comp rx=0 tx=0).stop 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c862b640 1 -- 172.21.15.31:0/1701321909 shutdown_connections 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c862b640 1 --2- 172.21.15.31:0/1701321909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c00d1fe0 0x7f40c00b2890 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c862b640 1 --2- 172.21.15.31:0/1701321909 >> v2:172.21.15.137:3300/0 conn(0x7f40c015d700 0x7f40c00d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c862b640 1 -- 172.21.15.31:0/1701321909 >> 172.21.15.31:0/1701321909 conn(0x7f40c00d32f0 msgr2=0x7f40c00d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.636+0000 7f40c862b640 1 -- 172.21.15.31:0/1701321909 shutdown_connections 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.637+0000 7f40c862b640 1 -- 172.21.15.31:0/1701321909 wait complete. 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 Processor -- start 2024-01-19T04:55:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 -- start start 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40c00d11b0 0x7f40c00b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c00d1fe0 0x7f40c015fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c015d700 0x7f40c00b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01601e0 con 0x7f40c015d700 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c83a0 con 0x7f40c00d1fe0 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c862b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c8680 con 0x7f40c00d11b0 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c6bc8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c015d700 0x7f40c00b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c6bc8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c015d700 0x7f40c00b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53930/0 (socket says 172.21.15.31:53930) 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.638+0000 7f40c6bc8640 1 -- 172.21.15.31:0/3642568985 learned_addr learned my addr 172.21.15.31:0/3642568985 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c5bc6640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c00d1fe0 0x7f40c015fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c6bc8640 1 -- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40c00d11b0 msgr2=0x7f40c00b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c6bc8640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40c00d11b0 0x7f40c00b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c6bc8640 1 -- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c00d1fe0 msgr2=0x7f40c015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c6bc8640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c00d1fe0 0x7f40c015fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c6bc8640 1 -- 172.21.15.31:0/3642568985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009b30 con 0x7f40c015d700 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c6bc8640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c015d700 0x7f40c00b5e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40b8036040 tx=0x7f40b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40b6ffd640 1 -- 172.21.15.31:0/3642568985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8009510 con 0x7f40c015d700 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40b6ffd640 1 -- 172.21.15.31:0/3642568985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b8007e50 con 0x7f40c015d700 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c00c8960 con 0x7f40c015d700 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40b6ffd640 1 -- 172.21.15.31:0/3642568985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b803e670 con 0x7f40c015d700 2024-01-19T04:55:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.639+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c00c8f20 con 0x7f40c015d700 2024-01-19T04:55:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.640+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c0074900 con 0x7f40c015d700 2024-01-19T04:55:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.640+0000 7f40b6ffd640 1 -- 172.21.15.31:0/3642568985 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f40b803e7d0 con 0x7f40c015d700 2024-01-19T04:55:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.641+0000 7f40b6ffd640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40a406af00 0x7f40a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.641+0000 7f40b6ffd640 1 -- 172.21.15.31:0/3642568985 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40b80abe40 con 0x7f40c015d700 2024-01-19T04:55:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.641+0000 7f40c63c7640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40a406af00 0x7f40a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.642+0000 7f40c63c7640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40a406af00 0x7f40a406d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40bc006fd0 tx=0x7f40bc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.645+0000 7f40b6ffd640 1 -- 172.21.15.31:0/3642568985 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f40b8076da0 con 0x7f40c015d700 2024-01-19T04:55:47.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.821+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40c00d16b0 con 0x7f40c015d700 2024-01-19T04:55:47.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.821+0000 7f40b6ffd640 1 -- 172.21.15.31:0/3642568985 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40b807a3c0 con 0x7f40c015d700 2024-01-19T04:55:47.821 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40a406af00 msgr2=0x7f40a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40a406af00 0x7f40a406d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40bc006fd0 tx=0x7f40bc008040 comp rx=0 tx=0).stop 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c015d700 msgr2=0x7f40c00b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c015d700 0x7f40c00b5e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40b8036040 tx=0x7f40b8004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 shutdown_connections 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f40a406af00 0x7f40a406d3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40c015d700 0x7f40c00b5e70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c00d1fe0 0x7f40c015fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 --2- 172.21.15.31:0/3642568985 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40c00d11b0 0x7f40c00b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 >> 172.21.15.31:0/3642568985 conn(0x7f40c00d32f0 msgr2=0x7f40c00d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 shutdown_connections 2024-01-19T04:55:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.823+0000 7f40c862b640 1 -- 172.21.15.31:0/3642568985 wait complete. 2024-01-19T04:55:47.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:47.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.938+0000 7fc65b2d3640 1 Processor -- start 2024-01-19T04:55:47.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.939+0000 7fc65b2d3640 1 -- start start 2024-01-19T04:55:47.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65b2d3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 0x7fc654071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65b2d3640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc6540cb4c0 0x7fc6540ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65b2d3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65415a780 con 0x7fc6540ca860 2024-01-19T04:55:47.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65b2d3640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc654072670 con 0x7fc6540cb4c0 2024-01-19T04:55:47.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65b2d3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540ce380 con 0x7fc65415a3a0 2024-01-19T04:55:47.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65886e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc6540ca860 0x7fc6540cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53798/0 (socket says 172.21.15.31:53798) 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65886e640 1 -- 172.21.15.31:0/1459345644 learned_addr learned my addr 172.21.15.31:0/1459345644 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367834419 0 0) 0x7fc65415a780 con 0x7fc6540ca860 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc659870640 1 --2- 172.21.15.31:0/1459345644 >> v2:172.21.15.137:3300/0 conn(0x7fc6540cb4c0 0x7fc6540ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc640003520 con 0x7fc6540ca860 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc65906f640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 0x7fc654071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1634443766 0 0) 0x7fc640003520 con 0x7fc6540ca860 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 msgr2=0x7fc654071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 0x7fc654071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 >> v2:172.21.15.137:3300/0 conn(0x7fc6540cb4c0 msgr2=0x7fc6540ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 --2- 172.21.15.31:0/1459345644 >> v2:172.21.15.137:3300/0 conn(0x7fc6540cb4c0 0x7fc6540ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540cdd70 con 0x7fc6540ca860 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fc648001670 con 0x7fc6540ca860 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 >> v1:172.21.15.31:6789/0 conn(0x7fc6540ca860 legacy=0x7fc6540cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640003970 0x7fc640003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6400047c0 0x7fc640004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 0x7fc640005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654072670 con 0x7fc640003970 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540ce380 con 0x7fc6400047c0 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.940+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64000b980 con 0x7fc65415a3a0 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc65906f640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640003970 0x7fc640003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc65886e640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6400047c0 0x7fc640004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc659870640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 0x7fc640005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc65906f640 1 -- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 msgr2=0x7fc640005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc65906f640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65415a3a0 0x7fc640005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc65906f640 1 -- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6400047c0 msgr2=0x7fc640004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc65906f640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6400047c0 0x7fc640004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.941+0000 7fc65906f640 1 -- 172.21.15.31:0/1459345644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640003520 con 0x7fc640003970 2024-01-19T04:55:47.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc65906f640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640003970 0x7fc640003d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc650009ac0 tx=0x7fc6400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88828f74455eca4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65002c6b0 con 0x7fc640003970 2024-01-19T04:55:47.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65002c810 con 0x7fc640003970 2024-01-19T04:55:47.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc647fff640 1 -- 172.21.15.31:0/1459345644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65002cba0 con 0x7fc640003970 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc65b2d3640 1 -- 172.21.15.31:0/1459345644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640003970 msgr2=0x7fc640003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc65b2d3640 1 --2- 172.21.15.31:0/1459345644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640003970 0x7fc640003d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc650009ac0 tx=0x7fc6400057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc659870640 1 -- 172.21.15.31:0/1459345644 reap_dead start 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc65b2d3640 1 -- 172.21.15.31:0/1459345644 shutdown_connections 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.942+0000 7fc65b2d3640 1 -- 172.21.15.31:0/1459345644 >> 172.21.15.31:0/1459345644 conn(0x7fc6540d0000 msgr2=0x7fc6540d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.943+0000 7fc65b2d3640 1 -- 172.21.15.31:0/1459345644 shutdown_connections 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.943+0000 7fc65b2d3640 1 -- 172.21.15.31:0/1459345644 wait complete. 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.944+0000 7fc65b2d3640 1 Processor -- start 2024-01-19T04:55:47.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.944+0000 7fc65b2d3640 1 -- start start 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65b2d3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc640003970 0x7fc654159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65b2d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540afab0 0x7fc6541ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65b2d3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc654156530 0x7fc6541569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65b2d3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654156ef0 con 0x7fc6540afab0 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65b2d3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541eeba0 con 0x7fc654156530 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65b2d3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540a22f0 con 0x7fc640003970 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540afab0 0x7fc6541ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540afab0 0x7fc6541ee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53958/0 (socket says 172.21.15.31:53958) 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 -- 172.21.15.31:0/465745650 learned_addr learned my addr 172.21.15.31:0/465745650 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65906f640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc640003970 0x7fc654159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc659870640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc654156530 0x7fc6541569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 -- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc640003970 msgr2=0x7fc654159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc640003970 0x7fc654159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 -- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc654156530 msgr2=0x7fc6541569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc654156530 0x7fc6541569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 -- 172.21.15.31:0/465745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540a28b0 con 0x7fc6540afab0 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65886e640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540afab0 0x7fc6541ee660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc648001aa0 tx=0x7fc6480077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc645ffb640 1 -- 172.21.15.31:0/465745650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648007e80 con 0x7fc6540afab0 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc645ffb640 1 -- 172.21.15.31:0/465745650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64800f9c0 con 0x7fc6540afab0 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc645ffb640 1 -- 172.21.15.31:0/465745650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64800fd50 con 0x7fc6540afab0 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.945+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6540b0160 con 0x7fc6540afab0 2024-01-19T04:55:47.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.946+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540a2540 con 0x7fc6540afab0 2024-01-19T04:55:47.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.947+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6540a26d0 con 0x7fc6540afab0 2024-01-19T04:55:47.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.947+0000 7fc645ffb640 1 -- 172.21.15.31:0/465745650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc64801d460 con 0x7fc6540afab0 2024-01-19T04:55:47.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.947+0000 7fc645ffb640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc63806aea0 0x7fc63806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:47.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.948+0000 7fc645ffb640 1 -- 172.21.15.31:0/465745650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc648089670 con 0x7fc6540afab0 2024-01-19T04:55:47.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.948+0000 7fc65906f640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc63806aea0 0x7fc63806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:47.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.948+0000 7fc65906f640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc63806aea0 0x7fc63806d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc650036000 tx=0x7fc650030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:47.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:47.952+0000 7fc645ffb640 1 -- 172.21.15.31:0/465745650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc648054680 con 0x7fc6540afab0 2024-01-19T04:55:48.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.131+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc654074900 con 0x7fc6540afab0 2024-01-19T04:55:48.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.132+0000 7fc645ffb640 1 -- 172.21.15.31:0/465745650 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc648057ca0 con 0x7fc6540afab0 2024-01-19T04:55:48.131 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:48.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc63806aea0 msgr2=0x7fc63806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc63806aea0 0x7fc63806d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc650036000 tx=0x7fc650030000 comp rx=0 tx=0).stop 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540afab0 msgr2=0x7fc6541ee660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540afab0 0x7fc6541ee660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc648001aa0 tx=0x7fc6480077a0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 shutdown_connections 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc63806aea0 0x7fc63806d360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc654156530 0x7fc6541569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540afab0 0x7fc6541ee660 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 --2- 172.21.15.31:0/465745650 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc640003970 0x7fc654159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 >> 172.21.15.31:0/465745650 conn(0x7fc6540d0000 msgr2=0x7fc6540cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 shutdown_connections 2024-01-19T04:55:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.134+0000 7fc65b2d3640 1 -- 172.21.15.31:0/465745650 wait complete. 2024-01-19T04:55:48.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:48.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.246+0000 7eff07490640 1 Processor -- start 2024-01-19T04:55:48.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.246+0000 7eff07490640 1 -- start start 2024-01-19T04:55:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.247+0000 7eff07490640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 0x7eff0013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.247+0000 7eff07490640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff000b2ee0 con 0x7eff000b1f40 2024-01-19T04:55:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.247+0000 7eff07490640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff000cba60 con 0x7eff0006d6b0 2024-01-19T04:55:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.247+0000 7eff07490640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff000b23d0 con 0x7eff0015b260 2024-01-19T04:55:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.247+0000 7eff04a2b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7eff000b1f40 0x7eff0013df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53800/0 (socket says 172.21.15.31:53800) 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.247+0000 7eff04a2b640 1 -- 172.21.15.31:0/3088289657 learned_addr learned my addr 172.21.15.31:0/3088289657 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.247+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639476252 0 0) 0x7eff000b2ee0 con 0x7eff000b1f40 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeec003520 con 0x7eff000b1f40 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff05a2d640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 0x7eff0013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 775915124 0 0) 0x7efeec003520 con 0x7eff000b1f40 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 msgr2=0x7eff0013e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 0x7eff0013e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 >> v1:172.21.15.137:6789/0 conn(0x7eff0006d6b0 legacy=0x7eff0006da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0015b6f0 con 0x7eff000b1f40 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7efef0001670 con 0x7eff000b1f40 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 >> v1:172.21.15.31:6789/0 conn(0x7eff000b1f40 legacy=0x7eff0013df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0039a0 0x7efeec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0047c0 0x7efeec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 0x7efeec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000b23d0 con 0x7efeec0047c0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec00bab0 con 0x7efeec0039a0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec00de80 con 0x7eff0015b260 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff04a2b640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0047c0 0x7efeec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff05a2d640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 0x7efeec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff0522c640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0039a0 0x7efeec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff04a2b640 1 -- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 msgr2=0x7efeec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff04a2b640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0015b260 0x7efeec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff04a2b640 1 -- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0039a0 msgr2=0x7efeec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff04a2b640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0039a0 0x7efeec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.248+0000 7eff04a2b640 1 -- 172.21.15.31:0/3088289657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec003520 con 0x7efeec0047c0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.249+0000 7eff04a2b640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0047c0 0x7efeec004c40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efef0001050 tx=0x7efeec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c24d8297d462434 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.249+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0005ad0 con 0x7efeec0047c0 2024-01-19T04:55:48.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.249+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef0005c30 con 0x7efeec0047c0 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.249+0000 7efef7fff640 1 -- 172.21.15.31:0/3088289657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef002c890 con 0x7efeec0047c0 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.250+0000 7eff07490640 1 -- 172.21.15.31:0/3088289657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0047c0 msgr2=0x7efeec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.250+0000 7eff07490640 1 --2- 172.21.15.31:0/3088289657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0047c0 0x7efeec004c40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efef0001050 tx=0x7efeec0057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.250+0000 7eff05a2d640 1 -- 172.21.15.31:0/3088289657 reap_dead start 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.250+0000 7eff07490640 1 -- 172.21.15.31:0/3088289657 shutdown_connections 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.250+0000 7eff07490640 1 -- 172.21.15.31:0/3088289657 >> 172.21.15.31:0/3088289657 conn(0x7eff0006a790 msgr2=0x7eff0006aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.250+0000 7eff07490640 1 -- 172.21.15.31:0/3088289657 shutdown_connections 2024-01-19T04:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.250+0000 7eff07490640 1 -- 172.21.15.31:0/3088289657 wait complete. 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.251+0000 7eff07490640 1 Processor -- start 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.251+0000 7eff07490640 1 -- start start 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff07490640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0047c0 0x7eff00159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff07490640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff001e08a0 0x7eff0013a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff07490640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0013b2d0 0x7eff0013b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff07490640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0013bcc0 con 0x7eff001e08a0 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff07490640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000cc5c0 con 0x7efeec0047c0 2024-01-19T04:55:48.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff07490640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0015a3f0 con 0x7eff0013b2d0 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff001e08a0 0x7eff0013a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff001e08a0 0x7eff0013a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53988/0 (socket says 172.21.15.31:53988) 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 -- 172.21.15.31:0/1665248422 learned_addr learned my addr 172.21.15.31:0/1665248422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff05a2d640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0013b2d0 0x7eff0013b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff0522c640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0047c0 0x7eff00159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 -- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0013b2d0 msgr2=0x7eff0013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0013b2d0 0x7eff0013b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 -- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0047c0 msgr2=0x7eff00159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0047c0 0x7eff00159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 -- 172.21.15.31:0/1665248422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0015a120 con 0x7eff001e08a0 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff0522c640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0047c0 0x7eff00159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff04a2b640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff001e08a0 0x7eff0013a230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efef0001050 tx=0x7efef0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7efef5ffb640 1 -- 172.21.15.31:0/1665248422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef002cca0 con 0x7eff001e08a0 2024-01-19T04:55:48.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7efef5ffb640 1 -- 172.21.15.31:0/1665248422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef002ce00 con 0x7eff001e08a0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff001e2ca0 con 0x7eff001e08a0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.252+0000 7efef5ffb640 1 -- 172.21.15.31:0/1665248422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0007270 con 0x7eff001e08a0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.253+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff001e3260 con 0x7eff001e08a0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.254+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0015a5c0 con 0x7eff001e08a0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.254+0000 7efef5ffb640 1 -- 172.21.15.31:0/1665248422 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efef0032070 con 0x7eff001e08a0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.255+0000 7efef5ffb640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efee406af00 0x7efee406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.255+0000 7efef5ffb640 1 -- 172.21.15.31:0/1665248422 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efef00aa830 con 0x7eff001e08a0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.255+0000 7eff0522c640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efee406af00 0x7efee406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.255+0000 7eff0522c640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efee406af00 0x7efee406d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff000ce5e0 tx=0x7efef8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.259+0000 7efef5ffb640 1 -- 172.21.15.31:0/1665248422 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efef0075840 con 0x7eff001e08a0 2024-01-19T04:55:48.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.436+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff001e3480 con 0x7eff001e08a0 2024-01-19T04:55:48.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.437+0000 7efef5ffb640 1 -- 172.21.15.31:0/1665248422 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7efef0045020 con 0x7eff001e08a0 2024-01-19T04:55:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:48.437 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efee406af00 msgr2=0x7efee406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efee406af00 0x7efee406d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff000ce5e0 tx=0x7efef8009040 comp rx=0 tx=0).stop 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff001e08a0 msgr2=0x7eff0013a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff001e08a0 0x7eff0013a230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efef0001050 tx=0x7efef0000fe0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 shutdown_connections 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efee406af00 0x7efee406d3c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff0013b2d0 0x7eff0013b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff001e08a0 0x7eff0013a230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 --2- 172.21.15.31:0/1665248422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeec0047c0 0x7eff00159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 >> 172.21.15.31:0/1665248422 conn(0x7eff0006a790 msgr2=0x7eff0013eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 shutdown_connections 2024-01-19T04:55:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.439+0000 7eff07490640 1 -- 172.21.15.31:0/1665248422 wait complete. 2024-01-19T04:55:48.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:48.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.558+0000 7f07bd7f5640 1 Processor -- start 2024-01-19T04:55:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.558+0000 7f07bd7f5640 1 -- start start 2024-01-19T04:55:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07bd7f5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f07b80cf6f0 0x7f07b80add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07bd7f5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07bd7f5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b80b0a70 con 0x7f07b80ccc70 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07bd7f5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b80b18d0 con 0x7f07b80cf6f0 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07bd7f5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80b0cc0 con 0x7f07b80cfad0 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07b67fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07b77fe640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f07b80ccc70 0x7f07b80b1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:60760/0 (socket says 172.21.15.31:60760) 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07b77fe640 1 -- 172.21.15.31:0/2169965439 learned_addr learned my addr 172.21.15.31:0/2169965439 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.559+0000 7f07b6ffd640 1 --2- 172.21.15.31:0/2169965439 >> v2:172.21.15.137:3300/0 conn(0x7f07b80cf6f0 0x7f07b80add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b5ffb640 1 -- 172.21.15.31:0/2169965439 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981309001 0 0) 0x7f07b80b0a70 con 0x7f07b80ccc70 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b5ffb640 1 -- 172.21.15.31:0/2169965439 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07a4003520 con 0x7f07b80ccc70 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b67fc640 1 -- 172.21.15.31:0/2169965439 >> v2:172.21.15.137:3300/0 conn(0x7f07b80cf6f0 msgr2=0x7f07b80add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b67fc640 1 --2- 172.21.15.31:0/2169965439 >> v2:172.21.15.137:3300/0 conn(0x7f07b80cf6f0 0x7f07b80add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b67fc640 1 -- 172.21.15.31:0/2169965439 >> v1:172.21.15.137:6789/0 conn(0x7f07b80ccc70 legacy=0x7f07b80b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b67fc640 1 -- 172.21.15.31:0/2169965439 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80b0fa0 con 0x7f07b80cfad0 2024-01-19T04:55:48.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b6ffd640 1 --2- 172.21.15.31:0/2169965439 >> v2:172.21.15.137:3300/0 conn(0x7f07b80cf6f0 0x7f07b80add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.560+0000 7f07b67fc640 1 --2- 172.21.15.31:0/2169965439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80cc730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07a00097a0 tx=0x7f07a002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea53f0fab0a2e2f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07b5ffb640 1 -- 172.21.15.31:0/2169965439 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0031830 con 0x7f07b80cfad0 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07b5ffb640 1 -- 172.21.15.31:0/2169965439 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a0031990 con 0x7f07b80cfad0 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07b5ffb640 1 -- 172.21.15.31:0/2169965439 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0031d20 con 0x7f07b80cfad0 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 -- 172.21.15.31:0/2169965439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 msgr2=0x7f07b80cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/2169965439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80cc730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07a00097a0 tx=0x7f07a002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 -- 172.21.15.31:0/2169965439 shutdown_connections 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/2169965439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80cc730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/2169965439 >> v2:172.21.15.137:3300/0 conn(0x7f07b80cf6f0 0x7f07b80add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 -- 172.21.15.31:0/2169965439 >> 172.21.15.31:0/2169965439 conn(0x7f07b808afe0 msgr2=0x7f07b80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 -- 172.21.15.31:0/2169965439 shutdown_connections 2024-01-19T04:55:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.561+0000 7f07bd7f5640 1 -- 172.21.15.31:0/2169965439 wait complete. 2024-01-19T04:55:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.562+0000 7f07bd7f5640 1 Processor -- start 2024-01-19T04:55:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07bd7f5640 1 -- start start 2024-01-19T04:55:48.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07bd7f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b80ccc70 0x7f07b8158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07bd7f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07b80cf6f0 0x7f07b80ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07bd7f5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07bd7f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80c52f0 con 0x7f07b80cf6f0 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07bd7f5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80c4ce0 con 0x7f07b80ccc70 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07bd7f5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80cacb0 con 0x7f07b80cfad0 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07b67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07b80cf6f0 0x7f07b80ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07b67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07b80cf6f0 0x7f07b80ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53990/0 (socket says 172.21.15.31:53990) 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.563+0000 7f07b67fc640 1 -- 172.21.15.31:0/234520704 learned_addr learned my addr 172.21.15.31:0/234520704 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b6ffd640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b80ccc70 0x7f07b8158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b77fe640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b67fc640 1 -- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 msgr2=0x7f07b80ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b67fc640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b67fc640 1 -- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b80ccc70 msgr2=0x7f07b8158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b67fc640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b80ccc70 0x7f07b8158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b67fc640 1 -- 172.21.15.31:0/234520704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8009a70 con 0x7f07b80cf6f0 2024-01-19T04:55:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07b67fc640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07b80cf6f0 0x7f07b80ca230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f07a002bee0 tx=0x7f07a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07af7fe640 1 -- 172.21.15.31:0/234520704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a003a630 con 0x7f07b80cf6f0 2024-01-19T04:55:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07af7fe640 1 -- 172.21.15.31:0/234520704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a003a7b0 con 0x7f07b80cf6f0 2024-01-19T04:55:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a0009400 con 0x7f07b80cf6f0 2024-01-19T04:55:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07af7fe640 1 -- 172.21.15.31:0/234520704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a003ab60 con 0x7f07b80cf6f0 2024-01-19T04:55:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.564+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b80c4f80 con 0x7f07b80cf6f0 2024-01-19T04:55:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.565+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b80c5460 con 0x7f07b80cf6f0 2024-01-19T04:55:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.566+0000 7f07af7fe640 1 -- 172.21.15.31:0/234520704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f07a003acc0 con 0x7f07b80cf6f0 2024-01-19T04:55:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.566+0000 7f07af7fe640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f079806aed0 0x7f079806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.566+0000 7f07af7fe640 1 -- 172.21.15.31:0/234520704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07a00ab170 con 0x7f07b80cf6f0 2024-01-19T04:55:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.566+0000 7f07b6ffd640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f079806aed0 0x7f079806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.567+0000 7f07b6ffd640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f079806aed0 0x7f079806d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07a80114e0 tx=0x7f07a800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.570+0000 7f07af7fe640 1 -- 172.21.15.31:0/234520704 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f07a00760f0 con 0x7f07b80cf6f0 2024-01-19T04:55:48.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.746+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07b8074900 con 0x7f07b80cf6f0 2024-01-19T04:55:48.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.746+0000 7f07af7fe640 1 -- 172.21.15.31:0/234520704 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07a0079710 con 0x7f07b80cf6f0 2024-01-19T04:55:48.746 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:48.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f079806aed0 msgr2=0x7f079806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f079806aed0 0x7f079806d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07a80114e0 tx=0x7f07a800b040 comp rx=0 tx=0).stop 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07b80cf6f0 msgr2=0x7f07b80ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07b80cf6f0 0x7f07b80ca230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f07a002bee0 tx=0x7f07a00046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 shutdown_connections 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f079806aed0 0x7f079806d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07b80cfad0 0x7f07b80ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07b80cf6f0 0x7f07b80ca230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 --2- 172.21.15.31:0/234520704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b80ccc70 0x7f07b8158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.748+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 >> 172.21.15.31:0/234520704 conn(0x7f07b808afe0 msgr2=0x7f07b80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.749+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 shutdown_connections 2024-01-19T04:55:48.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.749+0000 7f07bd7f5640 1 -- 172.21.15.31:0/234520704 wait complete. 2024-01-19T04:55:48.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:48.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.863+0000 7fe2c8f94640 1 Processor -- start 2024-01-19T04:55:48.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.863+0000 7fe2c8f94640 1 -- start start 2024-01-19T04:55:48.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c8f94640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2c4161860 0x7fe2c4168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c8f94640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c40cf9f0 con 0x7fe2c415d3a0 2024-01-19T04:55:48.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c8f94640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c40cfcd0 con 0x7fe2c40cf570 2024-01-19T04:55:48.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c8f94640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c40cffb0 con 0x7fe2c4161860 2024-01-19T04:55:48.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c2d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2c4161860 0x7fe2c4168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c259c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe2c415d3a0 0x7fe2c406c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53806/0 (socket says 172.21.15.31:53806) 2024-01-19T04:55:48.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c259c640 1 -- 172.21.15.31:0/4052752773 learned_addr learned my addr 172.21.15.31:0/4052752773 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:48.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c2d9d640 1 -- 172.21.15.31:0/4052752773 >> v1:172.21.15.137:6789/0 conn(0x7fe2c40cf570 legacy=0x7fe2c4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c2d9d640 1 -- 172.21.15.31:0/4052752773 >> v1:172.21.15.31:6789/0 conn(0x7fe2c415d3a0 legacy=0x7fe2c406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c2d9d640 1 -- 172.21.15.31:0/4052752773 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40d0290 con 0x7fe2c4161860 2024-01-19T04:55:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.864+0000 7fe2c159a640 1 -- 172.21.15.31:0/4052752773 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650836444 0 0) 0x7fe2c40cf9f0 con 0x7fe2c415d3a0 2024-01-19T04:55:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.865+0000 7fe2c2d9d640 1 --2- 172.21.15.31:0/4052752773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c4168490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009d90 tx=0x7fe2b402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d42995889ef5a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.865+0000 7fe2c159a640 1 -- 172.21.15.31:0/4052752773 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b402cbd0 con 0x7fe2c4161860 2024-01-19T04:55:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.865+0000 7fe2c159a640 1 -- 172.21.15.31:0/4052752773 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b402cd30 con 0x7fe2c4161860 2024-01-19T04:55:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.865+0000 7fe2c159a640 1 -- 172.21.15.31:0/4052752773 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b402cea0 con 0x7fe2c4161860 2024-01-19T04:55:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.866+0000 7fe2c8f94640 1 -- 172.21.15.31:0/4052752773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 msgr2=0x7fe2c4168490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.866+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/4052752773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c4168490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009d90 tx=0x7fe2b402be20 comp rx=0 tx=0).stop 2024-01-19T04:55:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.866+0000 7fe2c8f94640 1 -- 172.21.15.31:0/4052752773 shutdown_connections 2024-01-19T04:55:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.866+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/4052752773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c4168490 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.866+0000 7fe2c8f94640 1 -- 172.21.15.31:0/4052752773 >> 172.21.15.31:0/4052752773 conn(0x7fe2c40d1100 msgr2=0x7fe2c40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.866+0000 7fe2c8f94640 1 -- 172.21.15.31:0/4052752773 shutdown_connections 2024-01-19T04:55:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.866+0000 7fe2c8f94640 1 -- 172.21.15.31:0/4052752773 wait complete. 2024-01-19T04:55:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.867+0000 7fe2c8f94640 1 Processor -- start 2024-01-19T04:55:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.867+0000 7fe2c8f94640 1 -- start start 2024-01-19T04:55:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c8f94640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2c40cf570 0x7fe2c41f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c8f94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c415d3a0 0x7fe2c40c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c8f94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c40c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c8f94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40b1350 con 0x7fe2c4161860 2024-01-19T04:55:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c8f94640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40b1630 con 0x7fe2c415d3a0 2024-01-19T04:55:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c8f94640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40b1910 con 0x7fe2c40cf570 2024-01-19T04:55:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c40c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c40c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54002/0 (socket says 172.21.15.31:54002) 2024-01-19T04:55:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 -- 172.21.15.31:0/3762854118 learned_addr learned my addr 172.21.15.31:0/3762854118 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c259c640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2c40cf570 0x7fe2c41f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c1d9b640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c415d3a0 0x7fe2c40c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 -- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2c40cf570 msgr2=0x7fe2c41f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2c40cf570 0x7fe2c41f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 -- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c415d3a0 msgr2=0x7fe2c40c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c415d3a0 0x7fe2c40c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c2d9d640 1 -- 172.21.15.31:0/3762854118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4009a70 con 0x7fe2c4161860 2024-01-19T04:55:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c259c640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2c40cf570 0x7fe2c41f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.868+0000 7fe2c1d9b640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c415d3a0 0x7fe2c40c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.869+0000 7fe2c2d9d640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c40c6fa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009d90 tx=0x7fe2b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.869+0000 7fe2baffd640 1 -- 172.21.15.31:0/3762854118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b402cea0 con 0x7fe2c4161860 2024-01-19T04:55:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.869+0000 7fe2baffd640 1 -- 172.21.15.31:0/3762854118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b4040450 con 0x7fe2c4161860 2024-01-19T04:55:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.869+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c40b1bf0 con 0x7fe2c4161860 2024-01-19T04:55:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.869+0000 7fe2baffd640 1 -- 172.21.15.31:0/3762854118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40407e0 con 0x7fe2c4161860 2024-01-19T04:55:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.869+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c41f33d0 con 0x7fe2c4161860 2024-01-19T04:55:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.870+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c4074900 con 0x7fe2c4161860 2024-01-19T04:55:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.870+0000 7fe2baffd640 1 -- 172.21.15.31:0/3762854118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe2b4040940 con 0x7fe2c4161860 2024-01-19T04:55:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.871+0000 7fe2baffd640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2a406af00 0x7fe2a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.871+0000 7fe2baffd640 1 -- 172.21.15.31:0/3762854118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b40aba80 con 0x7fe2c4161860 2024-01-19T04:55:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.871+0000 7fe2c259c640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2a406af00 0x7fe2a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:48.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.871+0000 7fe2c259c640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2a406af00 0x7fe2a406d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40affa0 tx=0x7fe2bc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:48.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:48.875+0000 7fe2baffd640 1 -- 172.21.15.31:0/3762854118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe2b4076a10 con 0x7fe2c4161860 2024-01-19T04:55:49.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.053+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2c40af220 con 0x7fe2c4161860 2024-01-19T04:55:49.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.053+0000 7fe2baffd640 1 -- 172.21.15.31:0/3762854118 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2b407a030 con 0x7fe2c4161860 2024-01-19T04:55:49.053 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:49.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.055+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2a406af00 msgr2=0x7fe2a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.055+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2a406af00 0x7fe2a406d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40affa0 tx=0x7fe2bc009040 comp rx=0 tx=0).stop 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 msgr2=0x7fe2c40c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c40c6fa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009d90 tx=0x7fe2b4004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 shutdown_connections 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2a406af00 0x7fe2a406d3c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c4161860 0x7fe2c40c6fa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c415d3a0 0x7fe2c40c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 --2- 172.21.15.31:0/3762854118 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2c40cf570 0x7fe2c41f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 >> 172.21.15.31:0/3762854118 conn(0x7fe2c40d1100 msgr2=0x7fe2c40d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 shutdown_connections 2024-01-19T04:55:49.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.056+0000 7fe2c8f94640 1 -- 172.21.15.31:0/3762854118 wait complete. 2024-01-19T04:55:49.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.170+0000 7f44ea77d640 1 Processor -- start 2024-01-19T04:55:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.170+0000 7f44ea77d640 1 -- start start 2024-01-19T04:55:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44ea77d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f44e40cb2e0 0x7f44e415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44ea77d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e40716f0 0x7f44e415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44ea77d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e415ece0 con 0x7f44e415b510 2024-01-19T04:55:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44ea77d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e4071c20 con 0x7f44e40cb2e0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44ea77d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4071f00 con 0x7f44e40716f0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f44e40cb2e0 0x7f44e415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f44e40cb2e0 0x7f44e415e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54012/0 (socket says 172.21.15.31:54012) 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e3fff640 1 -- 172.21.15.31:0/2329013877 learned_addr learned my addr 172.21.15.31:0/2329013877 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e37fe640 1 --2- 172.21.15.31:0/2329013877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e40716f0 0x7f44e415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e3fff640 1 -- 172.21.15.31:0/2329013877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e40716f0 msgr2=0x7f44e415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e3fff640 1 --2- 172.21.15.31:0/2329013877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e40716f0 0x7f44e415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e3fff640 1 -- 172.21.15.31:0/2329013877 >> v1:172.21.15.137:6789/0 conn(0x7f44e415b510 legacy=0x7f44e406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.171+0000 7f44e3fff640 1 -- 172.21.15.31:0/2329013877 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40721e0 con 0x7f44e40cb2e0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.172+0000 7f44e3fff640 1 --2- 172.21.15.31:0/2329013877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40cb2e0 0x7f44e415e7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009870 tx=0x7f44d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a426448808241ce1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.172+0000 7f44e2ffd640 1 -- 172.21.15.31:0/2329013877 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8031640 con 0x7f44e40cb2e0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.172+0000 7f44e2ffd640 1 -- 172.21.15.31:0/2329013877 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d80317a0 con 0x7f44e40cb2e0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.172+0000 7f44e2ffd640 1 -- 172.21.15.31:0/2329013877 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8031b30 con 0x7f44e40cb2e0 2024-01-19T04:55:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 -- 172.21.15.31:0/2329013877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40cb2e0 msgr2=0x7f44e415e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 --2- 172.21.15.31:0/2329013877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40cb2e0 0x7f44e415e7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009870 tx=0x7f44d802b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 -- 172.21.15.31:0/2329013877 shutdown_connections 2024-01-19T04:55:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 --2- 172.21.15.31:0/2329013877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e40716f0 0x7f44e415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 --2- 172.21.15.31:0/2329013877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40cb2e0 0x7f44e415e7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 -- 172.21.15.31:0/2329013877 >> 172.21.15.31:0/2329013877 conn(0x7f44e406aa00 msgr2=0x7f44e406ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 -- 172.21.15.31:0/2329013877 shutdown_connections 2024-01-19T04:55:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.173+0000 7f44ea77d640 1 -- 172.21.15.31:0/2329013877 wait complete. 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.174+0000 7f44ea77d640 1 Processor -- start 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.174+0000 7f44ea77d640 1 -- start start 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44ea77d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40716f0 0x7f44e414bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44ea77d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44e40cb2e0 0x7f44e414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44ea77d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e415b510 0x7f44e406c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44ea77d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e414cd10 con 0x7f44e40716f0 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44ea77d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e414e220 con 0x7f44e40cb2e0 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44ea77d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e414c190 con 0x7f44e415b510 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40716f0 0x7f44e414bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40716f0 0x7f44e414bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54018/0 (socket says 172.21.15.31:54018) 2024-01-19T04:55:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 -- 172.21.15.31:0/1612091998 learned_addr learned my addr 172.21.15.31:0/1612091998 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 -- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e415b510 msgr2=0x7f44e406c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e415b510 0x7f44e406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 -- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44e40cb2e0 msgr2=0x7f44e414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44e40cb2e0 0x7f44e414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 -- 172.21.15.31:0/1612091998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d80094d0 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e3fff640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40716f0 0x7f44e414bb00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009870 tx=0x7f44d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e0ff9640 1 -- 172.21.15.31:0/1612091998 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d803ac60 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.175+0000 7f44e0ff9640 1 -- 172.21.15.31:0/1612091998 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d803adc0 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.176+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e414c470 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.176+0000 7f44e0ff9640 1 -- 172.21.15.31:0/1612091998 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8044630 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.176+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e414ca30 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.177+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e4074900 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.177+0000 7f44e0ff9640 1 -- 172.21.15.31:0/1612091998 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f44d802f050 con 0x7f44e40716f0 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.177+0000 7f44e0ff9640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f44d006aed0 0x7f44d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.177+0000 7f44e0ff9640 1 -- 172.21.15.31:0/1612091998 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44d80ab4c0 con 0x7f44e40716f0 2024-01-19T04:55:49.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.178+0000 7f44e37fe640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f44d006aed0 0x7f44d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.178+0000 7f44e37fe640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f44d006aed0 0x7f44d006d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44e4071b50 tx=0x7f44d4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.182+0000 7f44e0ff9640 1 -- 172.21.15.31:0/1612091998 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f44d8076420 con 0x7f44e40716f0 2024-01-19T04:55:49.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.359+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44e40b2e80 con 0x7f44e40716f0 2024-01-19T04:55:49.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.360+0000 7f44e0ff9640 1 -- 172.21.15.31:0/1612091998 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44d8079a40 con 0x7f44e40716f0 2024-01-19T04:55:49.359 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:49.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f44d006aed0 msgr2=0x7f44d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f44d006aed0 0x7f44d006d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44e4071b50 tx=0x7f44d4008040 comp rx=0 tx=0).stop 2024-01-19T04:55:49.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40716f0 msgr2=0x7f44e414bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40716f0 0x7f44e414bb00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009870 tx=0x7f44d80046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 shutdown_connections 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f44d006aed0 0x7f44d006d390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44e415b510 0x7f44e406c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44e40cb2e0 0x7f44e414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 --2- 172.21.15.31:0/1612091998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44e40716f0 0x7f44e414bb00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.361+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 >> 172.21.15.31:0/1612091998 conn(0x7f44e406aa00 msgr2=0x7f44e415b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.362+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 shutdown_connections 2024-01-19T04:55:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.362+0000 7f44ea77d640 1 -- 172.21.15.31:0/1612091998 wait complete. 2024-01-19T04:55:49.405 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2024-01-19T04:55:49.405 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-19T04:55:49.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.507+0000 7fc736018640 1 Processor -- start 2024-01-19T04:55:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.507+0000 7fc736018640 1 -- start start 2024-01-19T04:55:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc736018640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc7300d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc736018640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7300d0ef0 0x7fc7300d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc736018640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7300d06b0 con 0x7fc730160bb0 2024-01-19T04:55:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc736018640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7300d18c0 con 0x7fc7300d0ef0 2024-01-19T04:55:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc736018640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300d1b90 con 0x7fc7300cfd10 2024-01-19T04:55:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7300d0ef0 0x7fc7300d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7300d0ef0 0x7fc7300d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48184/0 (socket says 172.21.15.31:48184) 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72ffff640 1 -- 172.21.15.31:0/180281399 learned_addr learned my addr 172.21.15.31:0/180281399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72effd640 1 --2- 172.21.15.31:0/180281399 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc7300d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72ffff640 1 -- 172.21.15.31:0/180281399 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 msgr2=0x7fc7300d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72ffff640 1 --2- 172.21.15.31:0/180281399 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc7300d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72ffff640 1 -- 172.21.15.31:0/180281399 >> v1:172.21.15.137:6789/0 conn(0x7fc730160bb0 legacy=0x7fc7300b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72ffff640 1 -- 172.21.15.31:0/180281399 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300d1e70 con 0x7fc7300d0ef0 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.508+0000 7fc72effd640 1 --2- 172.21.15.31:0/180281399 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc7300d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.509+0000 7fc72ffff640 1 --2- 172.21.15.31:0/180281399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7300d0ef0 0x7fc7300d1380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc728009e50 tx=0x7fc72802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c731aaec7ed5caad server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.509+0000 7fc72e7fc640 1 -- 172.21.15.31:0/180281399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72802cb00 con 0x7fc7300d0ef0 2024-01-19T04:55:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.509+0000 7fc72e7fc640 1 -- 172.21.15.31:0/180281399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72802cc60 con 0x7fc7300d0ef0 2024-01-19T04:55:49.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.509+0000 7fc72e7fc640 1 -- 172.21.15.31:0/180281399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7280343f0 con 0x7fc7300d0ef0 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 -- 172.21.15.31:0/180281399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7300d0ef0 msgr2=0x7fc7300d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 --2- 172.21.15.31:0/180281399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7300d0ef0 0x7fc7300d1380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc728009e50 tx=0x7fc72802bf60 comp rx=0 tx=0).stop 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 -- 172.21.15.31:0/180281399 shutdown_connections 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 --2- 172.21.15.31:0/180281399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7300d0ef0 0x7fc7300d1380 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 --2- 172.21.15.31:0/180281399 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc7300d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 -- 172.21.15.31:0/180281399 >> 172.21.15.31:0/180281399 conn(0x7fc73009c600 msgr2=0x7fc7300d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 -- 172.21.15.31:0/180281399 shutdown_connections 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.510+0000 7fc736018640 1 -- 172.21.15.31:0/180281399 wait complete. 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.511+0000 7fc736018640 1 Processor -- start 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.511+0000 7fc736018640 1 -- start start 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc736018640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc73015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc736018640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7300d0ef0 0x7fc7300c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc736018640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc730160bb0 0x7fc7300c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc736018640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c7dc0 con 0x7fc730160bb0 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc736018640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300b6da0 con 0x7fc7300d0ef0 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc736018640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73015a9b0 con 0x7fc7300cfd10 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc730160bb0 0x7fc7300c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc730160bb0 0x7fc7300c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48194/0 (socket says 172.21.15.31:48194) 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 -- 172.21.15.31:0/3872265422 learned_addr learned my addr 172.21.15.31:0/3872265422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72f7fe640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc73015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72effd640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7300d0ef0 0x7fc7300c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 -- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 msgr2=0x7fc73015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc73015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 -- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7300d0ef0 msgr2=0x7fc7300c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7300d0ef0 0x7fc7300c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 -- 172.21.15.31:0/3872265422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728009b30 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72effd640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7300d0ef0 0x7fc7300c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.512+0000 7fc72ffff640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc730160bb0 0x7fc7300c81e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc728036040 tx=0x7fc728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.513+0000 7fc717fff640 1 -- 172.21.15.31:0/3872265422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728009510 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.513+0000 7fc717fff640 1 -- 172.21.15.31:0/3872265422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72803e450 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.513+0000 7fc717fff640 1 -- 172.21.15.31:0/3872265422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72803e800 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.513+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7300b67e0 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.513+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7300c79e0 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.514+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7300c7f00 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.514+0000 7fc717fff640 1 -- 172.21.15.31:0/3872265422 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc72803e960 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.514+0000 7fc717fff640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71c06aed0 0x7fc71c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.515+0000 7fc717fff640 1 -- 172.21.15.31:0/3872265422 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7280ac390 con 0x7fc730160bb0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.515+0000 7fc72f7fe640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71c06aed0 0x7fc71c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.515+0000 7fc72f7fe640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71c06aed0 0x7fc71c06d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc720001640 tx=0x7fc720009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.519+0000 7fc717fff640 1 -- 172.21.15.31:0/3872265422 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc7300c7f00 con 0x7fc730160bb0 2024-01-19T04:55:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.700+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7300c7f00 con 0x7fc730160bb0 2024-01-19T04:55:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.701+0000 7fc717fff640 1 -- 172.21.15.31:0/3872265422 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fc7300c7f00 con 0x7fc730160bb0 2024-01-19T04:55:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:49.701 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71c06aed0 msgr2=0x7fc71c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71c06aed0 0x7fc71c06d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc720001640 tx=0x7fc720009040 comp rx=0 tx=0).stop 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc730160bb0 msgr2=0x7fc7300c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc730160bb0 0x7fc7300c81e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc728036040 tx=0x7fc728004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 shutdown_connections 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71c06aed0 0x7fc71c06d390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc730160bb0 0x7fc7300c81e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7300d0ef0 0x7fc7300c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 --2- 172.21.15.31:0/3872265422 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7300cfd10 0x7fc73015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 >> 172.21.15.31:0/3872265422 conn(0x7fc73009c600 msgr2=0x7fc73015e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:49.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 shutdown_connections 2024-01-19T04:55:49.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.703+0000 7fc736018640 1 -- 172.21.15.31:0/3872265422 wait complete. 2024-01-19T04:55:49.714 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:49.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:49.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa07217c640 1 Processor -- start 2024-01-19T04:55:49.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa07217c640 1 -- start start 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa07217c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa06c0d1820 0x7fa06c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa07217c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c0b0450 con 0x7fa06c15d570 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa07217c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c0cb760 con 0x7fa06c06c6e0 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa07217c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c0cba40 con 0x7fa06c0d1820 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa06b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa06c15d570 0x7fa06c06bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54958/0 (socket says 172.21.15.31:54958) 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.819+0000 7fa06b7fe640 1 -- 172.21.15.31:0/1356566980 learned_addr learned my addr 172.21.15.31:0/1356566980 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255284353 0 0) 0x7fa06c0b0450 con 0x7fa06c15d570 2024-01-19T04:55:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa054003520 con 0x7fa06c15d570 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06bfff640 1 --2- 172.21.15.31:0/1356566980 >> v2:172.21.15.137:3300/0 conn(0x7fa06c0d1820 0x7fa06c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2855098549 0 0) 0x7fa054003520 con 0x7fa06c15d570 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 >> v2:172.21.15.137:3300/0 conn(0x7fa06c0d1820 msgr2=0x7fa06c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 --2- 172.21.15.31:0/1356566980 >> v2:172.21.15.137:3300/0 conn(0x7fa06c0d1820 0x7fa06c0cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 >> v1:172.21.15.137:6789/0 conn(0x7fa06c06c6e0 legacy=0x7fa06c0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c0ce630 con 0x7fa06c15d570 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fa05c001670 con 0x7fa06c15d570 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 >> v1:172.21.15.31:6789/0 conn(0x7fa06c15d570 legacy=0x7fa06c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054003970 0x7fa054003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0540047c0 0x7fa054004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa054005180 0x7fa05400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0cba40 con 0x7fa054005180 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05400c120 con 0x7fa0540047c0 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05400c2e0 con 0x7fa054003970 2024-01-19T04:55:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06bfff640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa054005180 0x7fa05400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06affd640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0540047c0 0x7fa054004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.820+0000 7fa06b7fe640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054003970 0x7fa054003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06bfff640 1 -- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054003970 msgr2=0x7fa054003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06bfff640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054003970 0x7fa054003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06bfff640 1 -- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0540047c0 msgr2=0x7fa054004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06bfff640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0540047c0 0x7fa054004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06bfff640 1 -- 172.21.15.31:0/1356566980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa054003520 con 0x7fa054005180 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06bfff640 1 -- 172.21.15.31:0/1356566980 reap_dead start 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06bfff640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa054005180 0x7fa05400b980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa064009f90 tx=0x7fa0540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802ccdf1fa586b53 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06402cc20 con 0x7fa054005180 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400f040 con 0x7fa054005180 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.821+0000 7fa06a7fc640 1 -- 172.21.15.31:0/1356566980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064003990 con 0x7fa054005180 2024-01-19T04:55:49.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.822+0000 7fa07217c640 1 -- 172.21.15.31:0/1356566980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa054005180 msgr2=0x7fa05400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.822+0000 7fa07217c640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa054005180 0x7fa05400b980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa064009f90 tx=0x7fa0540057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.822+0000 7fa07217c640 1 -- 172.21.15.31:0/1356566980 shutdown_connections 2024-01-19T04:55:49.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.822+0000 7fa07217c640 1 --2- 172.21.15.31:0/1356566980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa054005180 0x7fa05400b980 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.822+0000 7fa07217c640 1 -- 172.21.15.31:0/1356566980 >> 172.21.15.31:0/1356566980 conn(0x7fa06c08afe0 msgr2=0x7fa06c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:49.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.822+0000 7fa07217c640 1 -- 172.21.15.31:0/1356566980 shutdown_connections 2024-01-19T04:55:49.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.822+0000 7fa07217c640 1 -- 172.21.15.31:0/1356566980 wait complete. 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.823+0000 7fa07217c640 1 Processor -- start 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.823+0000 7fa07217c640 1 -- start start 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa07217c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054005180 0x7fa06c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa07217c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa06c0b4870 0x7fa06c0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa07217c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa06c0c89e0 0x7fa06c1f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa07217c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0c6320 con 0x7fa06c0b4870 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa07217c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0ca990 con 0x7fa06c0c89e0 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa07217c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0c8e60 con 0x7fa054005180 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa06c0b4870 0x7fa06c0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa06c0b4870 0x7fa06c0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48216/0 (socket says 172.21.15.31:48216) 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 -- 172.21.15.31:0/2367742437 learned_addr learned my addr 172.21.15.31:0/2367742437 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:49.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06bfff640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa06c0c89e0 0x7fa06c1f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06b7fe640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054005180 0x7fa06c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 -- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054005180 msgr2=0x7fa06c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054005180 0x7fa06c15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 -- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa06c0c89e0 msgr2=0x7fa06c1f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa06c0c89e0 0x7fa06c1f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.824+0000 7fa06affd640 1 -- 172.21.15.31:0/2367742437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c1f39b0 con 0x7fa06c0b4870 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.825+0000 7fa06affd640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa06c0b4870 0x7fa06c0c84a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa06000f620 tx=0x7fa06000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.825+0000 7fa053fff640 1 -- 172.21.15.31:0/2367742437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0600052e0 con 0x7fa06c0b4870 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.825+0000 7fa053fff640 1 -- 172.21.15.31:0/2367742437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa060005440 con 0x7fa06c0b4870 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.825+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c1f3c90 con 0x7fa06c0b4870 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.825+0000 7fa053fff640 1 -- 172.21.15.31:0/2367742437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060003810 con 0x7fa06c0b4870 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.825+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c0c5f40 con 0x7fa06c0b4870 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.826+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c074900 con 0x7fa06c0b4870 2024-01-19T04:55:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.826+0000 7fa053fff640 1 -- 172.21.15.31:0/2367742437 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa060003970 con 0x7fa06c0b4870 2024-01-19T04:55:49.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.827+0000 7fa053fff640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa05806af00 0x7fa05806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:49.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.827+0000 7fa053fff640 1 -- 172.21.15.31:0/2367742437 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa060089400 con 0x7fa06c0b4870 2024-01-19T04:55:49.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.827+0000 7fa06b7fe640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa05806af00 0x7fa05806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:49.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.827+0000 7fa06b7fe640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa05806af00 0x7fa05806d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa05c001050 tx=0x7fa05c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:49.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:49.831+0000 7fa053fff640 1 -- 172.21.15.31:0/2367742437 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa0600542c0 con 0x7fa06c0b4870 2024-01-19T04:55:50.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.008+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa06c0ce920 con 0x7fa06c0b4870 2024-01-19T04:55:50.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.008+0000 7fa053fff640 1 -- 172.21.15.31:0/2367742437 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0600578e0 con 0x7fa06c0b4870 2024-01-19T04:55:50.008 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa05806af00 msgr2=0x7fa05806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa05806af00 0x7fa05806d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa05c001050 tx=0x7fa05c009040 comp rx=0 tx=0).stop 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa06c0b4870 msgr2=0x7fa06c0c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa06c0b4870 0x7fa06c0c84a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa06000f620 tx=0x7fa06000fae0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 shutdown_connections 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa05806af00 0x7fa05806d3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa06c0c89e0 0x7fa06c1f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa06c0b4870 0x7fa06c0c84a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 --2- 172.21.15.31:0/2367742437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa054005180 0x7fa06c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 >> 172.21.15.31:0/2367742437 conn(0x7fa06c08afe0 msgr2=0x7fa06c0add70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 shutdown_connections 2024-01-19T04:55:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.010+0000 7fa07217c640 1 -- 172.21.15.31:0/2367742437 wait complete. 2024-01-19T04:55:50.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:50.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.125+0000 7f35ed4a6640 1 Processor -- start 2024-01-19T04:55:50.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.125+0000 7f35ed4a6640 1 -- start start 2024-01-19T04:55:50.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35ed4a6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 0x7f35e80d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35ed4a6640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f35e80d3ea0 0x7f35e80b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35ed4a6640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e80cf160 con 0x7f35e815d720 2024-01-19T04:55:50.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35ed4a6640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e80ce8c0 con 0x7f35e80d3ea0 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35ed4a6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80ceba0 con 0x7f35e80d1c20 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35e6ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f35e815d720 0x7f35e80d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54964/0 (socket says 172.21.15.31:54964) 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35e6ffd640 1 -- 172.21.15.31:0/3549445767 learned_addr learned my addr 172.21.15.31:0/3549445767 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139872312 0 0) 0x7f35e80cf160 con 0x7f35e815d720 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35e67fc640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 0x7f35e80d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d4003520 con 0x7f35e815d720 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.126+0000 7f35e77fe640 1 --2- 172.21.15.31:0/3549445767 >> v2:172.21.15.137:3300/0 conn(0x7f35e80d3ea0 0x7f35e80b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651046702 0 0) 0x7f35d4003520 con 0x7f35e815d720 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 msgr2=0x7f35e80d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 0x7f35e80d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 >> v2:172.21.15.137:3300/0 conn(0x7f35e80d3ea0 msgr2=0x7f35e80b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 --2- 172.21.15.31:0/3549445767 >> v2:172.21.15.137:3300/0 conn(0x7f35e80d3ea0 0x7f35e80b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80cee80 con 0x7f35e815d720 2024-01-19T04:55:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f35d80019f0 con 0x7f35e815d720 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 >> v1:172.21.15.31:6789/0 conn(0x7f35e815d720 legacy=0x7f35e80d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 0x7f35d4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d4003ef0 0x7f35d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4004d00 0x7f35d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80ce8c0 con 0x7f35d4003ef0 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80ceba0 con 0x7f35d4004d00 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400ba60 con 0x7f35e80d1c20 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e67fc640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d4003ef0 0x7f35d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e6ffd640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 0x7f35d4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e77fe640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4004d00 0x7f35d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e67fc640 1 -- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 msgr2=0x7f35d4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e67fc640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35e80d1c20 0x7f35d4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e67fc640 1 -- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4004d00 msgr2=0x7f35d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e67fc640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4004d00 0x7f35d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.127+0000 7f35e67fc640 1 -- 172.21.15.31:0/3549445767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4003520 con 0x7f35d4003ef0 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.128+0000 7f35e67fc640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d4003ef0 0x7f35d40047c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009ac0 tx=0x7f35d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e6ce7d68645a64 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.128+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d002c6b0 con 0x7f35d4003ef0 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.128+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d002c810 con 0x7f35d4003ef0 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.128+0000 7f35e5ffb640 1 -- 172.21.15.31:0/3549445767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d002cba0 con 0x7f35d4003ef0 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.129+0000 7f35ed4a6640 1 -- 172.21.15.31:0/3549445767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d4003ef0 msgr2=0x7f35d40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.129+0000 7f35ed4a6640 1 --2- 172.21.15.31:0/3549445767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d4003ef0 0x7f35d40047c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009ac0 tx=0x7f35d40057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.129+0000 7f35e77fe640 1 -- 172.21.15.31:0/3549445767 reap_dead start 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.129+0000 7f35ed4a6640 1 -- 172.21.15.31:0/3549445767 shutdown_connections 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.129+0000 7f35ed4a6640 1 -- 172.21.15.31:0/3549445767 >> 172.21.15.31:0/3549445767 conn(0x7f35e80d3310 msgr2=0x7f35e80d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.129+0000 7f35ed4a6640 1 -- 172.21.15.31:0/3549445767 shutdown_connections 2024-01-19T04:55:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.129+0000 7f35ed4a6640 1 -- 172.21.15.31:0/3549445767 wait complete. 2024-01-19T04:55:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.130+0000 7f35ed4a6640 1 Processor -- start 2024-01-19T04:55:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.130+0000 7f35ed4a6640 1 -- start start 2024-01-19T04:55:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35ed4a6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35d4003ef0 0x7f35e80c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35ed4a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e80ca3e0 0x7f35e80ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35ed4a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e80b3ed0 0x7f35e80b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35ed4a6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80b48c0 con 0x7f35e80ca3e0 2024-01-19T04:55:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35ed4a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80cab50 con 0x7f35e80b3ed0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35ed4a6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80ca870 con 0x7f35d4003ef0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e80ca3e0 0x7f35e80ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e80ca3e0 0x7f35e80ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48234/0 (socket says 172.21.15.31:48234) 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 -- 172.21.15.31:0/65726675 learned_addr learned my addr 172.21.15.31:0/65726675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e6ffd640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35d4003ef0 0x7f35e80c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e77fe640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e80b3ed0 0x7f35e80b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 -- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35d4003ef0 msgr2=0x7f35e80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35d4003ef0 0x7f35e80c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 -- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e80b3ed0 msgr2=0x7f35e80b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e80b3ed0 0x7f35e80b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 -- 172.21.15.31:0/65726675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e81e5720 con 0x7f35e80ca3e0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.131+0000 7f35e67fc640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e80ca3e0 0x7f35e80ccbd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009ac0 tx=0x7f35d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.132+0000 7f35df7fe640 1 -- 172.21.15.31:0/65726675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0003810 con 0x7f35e80ca3e0 2024-01-19T04:55:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.132+0000 7f35df7fe640 1 -- 172.21.15.31:0/65726675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d0003970 con 0x7f35e80ca3e0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.132+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e81e5a00 con 0x7f35e80ca3e0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.132+0000 7f35df7fe640 1 -- 172.21.15.31:0/65726675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0003d20 con 0x7f35e80ca3e0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.132+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e80c9700 con 0x7f35e80ca3e0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.133+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e8074900 con 0x7f35e80ca3e0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.133+0000 7f35df7fe640 1 -- 172.21.15.31:0/65726675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f35d0040450 con 0x7f35e80ca3e0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.134+0000 7f35df7fe640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35c806af00 0x7f35c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.134+0000 7f35df7fe640 1 -- 172.21.15.31:0/65726675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35d00ab390 con 0x7f35e80ca3e0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.134+0000 7f35e6ffd640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35c806af00 0x7f35c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.134+0000 7f35e6ffd640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35c806af00 0x7f35c806d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35d8002070 tx=0x7f35d8006040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.138+0000 7f35df7fe640 1 -- 172.21.15.31:0/65726675 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f35d0076280 con 0x7f35e80ca3e0 2024-01-19T04:55:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.319+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35e80b39b0 con 0x7f35e80ca3e0 2024-01-19T04:55:50.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.319+0000 7f35df7fe640 1 -- 172.21.15.31:0/65726675 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35d00044e0 con 0x7f35e80ca3e0 2024-01-19T04:55:50.319 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35c806af00 msgr2=0x7f35c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35c806af00 0x7f35c806d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35d8002070 tx=0x7f35d8006040 comp rx=0 tx=0).stop 2024-01-19T04:55:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e80ca3e0 msgr2=0x7f35e80ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e80ca3e0 0x7f35e80ccbd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009ac0 tx=0x7f35d0004730 comp rx=0 tx=0).stop 2024-01-19T04:55:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 shutdown_connections 2024-01-19T04:55:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35c806af00 0x7f35c806d3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e80b3ed0 0x7f35e80b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e80ca3e0 0x7f35e80ccbd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 --2- 172.21.15.31:0/65726675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35d4003ef0 0x7f35e80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 >> 172.21.15.31:0/65726675 conn(0x7f35e80d3310 msgr2=0x7f35e806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 shutdown_connections 2024-01-19T04:55:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.321+0000 7f35ed4a6640 1 -- 172.21.15.31:0/65726675 wait complete. 2024-01-19T04:55:50.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.436+0000 7f7e0d1b0640 1 Processor -- start 2024-01-19T04:55:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.436+0000 7f7e0d1b0640 1 -- start start 2024-01-19T04:55:50.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0d1b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e0815d720 0x7f7e080d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0d1b0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7e080d3ea0 0x7f7e080d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0d1b0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e080cf160 con 0x7f7e080d1a50 2024-01-19T04:55:50.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0d1b0640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e080d1f00 con 0x7f7e080d3ea0 2024-01-19T04:55:50.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0d1b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080ce8c0 con 0x7f7e0815d720 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0659c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7e080d3ea0 0x7f7e080d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e06d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e0815d720 0x7f7e080d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0759e640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f7e080d1a50 0x7f7e080cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:41268/0 (socket says 172.21.15.31:41268) 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0659c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7e080d3ea0 0x7f7e080d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:41272/0 (socket says 172.21.15.31:41272) 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e06d9d640 1 --2- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e0815d720 0x7f7e080d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:46170/0 (socket says 172.21.15.31:46170) 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.437+0000 7f7e0759e640 1 -- 172.21.15.31:0/1408019451 learned_addr learned my addr 172.21.15.31:0/1408019451 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.438+0000 7f7e0659c640 1 -- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e0815d720 msgr2=0x7f7e080d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.438+0000 7f7e0659c640 1 --2- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e0815d720 0x7f7e080d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.438+0000 7f7e0659c640 1 -- 172.21.15.31:0/1408019451 >> v1:172.21.15.137:6789/0 conn(0x7f7e080d1a50 legacy=0x7f7e080cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.438+0000 7f7e0659c640 1 -- 172.21.15.31:0/1408019451 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080ceba0 con 0x7f7e080d3ea0 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.438+0000 7f7e05d9b640 1 -- 172.21.15.31:0/1408019451 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291963908 0 0) 0x7f7e080cf160 con 0x7f7e080d1a50 2024-01-19T04:55:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.438+0000 7f7e06d9d640 1 --2- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e0815d720 0x7f7e080d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0659c640 1 --2- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e080d3ea0 0x7f7e080d1510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7df00097a0 tx=0x7f7df002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9cd44c9ff8cb04a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e05d9b640 1 -- 172.21.15.31:0/1408019451 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df002f050 con 0x7f7e080d3ea0 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e05d9b640 1 -- 172.21.15.31:0/1408019451 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df0031700 con 0x7f7e080d3ea0 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e05d9b640 1 -- 172.21.15.31:0/1408019451 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0031a90 con 0x7f7e080d3ea0 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e080d3ea0 msgr2=0x7f7e080d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e080d3ea0 0x7f7e080d1510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7df00097a0 tx=0x7f7df002b7a0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/1408019451 shutdown_connections 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e080d3ea0 0x7f7e080d1510 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/1408019451 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e0815d720 0x7f7e080d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/1408019451 >> 172.21.15.31:0/1408019451 conn(0x7f7e080d3310 msgr2=0x7f7e080d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/1408019451 shutdown_connections 2024-01-19T04:55:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.439+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/1408019451 wait complete. 2024-01-19T04:55:50.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 Processor -- start 2024-01-19T04:55:50.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 -- start start 2024-01-19T04:55:50.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e080d1a50 0x7f7e08158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e080d3ea0 0x7f7e081601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0815d720 0x7f7e08160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080b6770 con 0x7f7e080d3ea0 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08158980 con 0x7f7e0815d720 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.441+0000 7f7e0d1b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08160c70 con 0x7f7e080d1a50 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e080d3ea0 0x7f7e081601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e080d3ea0 0x7f7e081601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48244/0 (socket says 172.21.15.31:48244) 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 -- 172.21.15.31:0/898760895 learned_addr learned my addr 172.21.15.31:0/898760895 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e06d9d640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e080d1a50 0x7f7e08158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0759e640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0815d720 0x7f7e08160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 -- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e080d1a50 msgr2=0x7f7e08158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e080d1a50 0x7f7e08158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 -- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0815d720 msgr2=0x7f7e08160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0815d720 0x7f7e08160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 -- 172.21.15.31:0/898760895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00009400 con 0x7f7e080d3ea0 2024-01-19T04:55:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0759e640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0815d720 0x7f7e08160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0659c640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e080d3ea0 0x7f7e081601f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7df00097a0 tx=0x7f7df00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7dff7fe640 1 -- 172.21.15.31:0/898760895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0033b30 con 0x7f7e080d3ea0 2024-01-19T04:55:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7dff7fe640 1 -- 172.21.15.31:0/898760895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df0033c90 con 0x7f7e080d3ea0 2024-01-19T04:55:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df0009400 con 0x7f7e080d3ea0 2024-01-19T04:55:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7dff7fe640 1 -- 172.21.15.31:0/898760895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df003d470 con 0x7f7e080d3ea0 2024-01-19T04:55:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.442+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e080b60b0 con 0x7f7e080d3ea0 2024-01-19T04:55:50.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.443+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e080b5f10 con 0x7f7e080d3ea0 2024-01-19T04:55:50.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.444+0000 7f7dff7fe640 1 -- 172.21.15.31:0/898760895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7df003a040 con 0x7f7e080d3ea0 2024-01-19T04:55:50.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.444+0000 7f7dff7fe640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7de806aed0 0x7f7de806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.444+0000 7f7dff7fe640 1 -- 172.21.15.31:0/898760895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7df00ab250 con 0x7f7e080d3ea0 2024-01-19T04:55:50.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.444+0000 7f7e06d9d640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7de806aed0 0x7f7de806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.445+0000 7f7e06d9d640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7de806aed0 0x7f7de806d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009c30 tx=0x7f7e00009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.448+0000 7f7dff7fe640 1 -- 172.21.15.31:0/898760895 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7e080b5f10 con 0x7f7e080d3ea0 2024-01-19T04:55:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.628+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e080b5f10 con 0x7f7e080d3ea0 2024-01-19T04:55:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.628+0000 7f7dff7fe640 1 -- 172.21.15.31:0/898760895 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e080b5f10 con 0x7f7e080d3ea0 2024-01-19T04:55:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7de806aed0 msgr2=0x7f7de806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7de806aed0 0x7f7de806d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009c30 tx=0x7f7e00009190 comp rx=0 tx=0).stop 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e080d3ea0 msgr2=0x7f7e081601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e080d3ea0 0x7f7e081601f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7df00097a0 tx=0x7f7df00046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 shutdown_connections 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7de806aed0 0x7f7de806d390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0815d720 0x7f7e08160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e080d3ea0 0x7f7e081601f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 --2- 172.21.15.31:0/898760895 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e080d1a50 0x7f7e08158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 >> 172.21.15.31:0/898760895 conn(0x7f7e080d3310 msgr2=0x7f7e080b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 shutdown_connections 2024-01-19T04:55:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.630+0000 7f7e0d1b0640 1 -- 172.21.15.31:0/898760895 wait complete. 2024-01-19T04:55:50.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-01-19T04:55:50.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.743+0000 7f0daf6af640 1 Processor -- start 2024-01-19T04:55:50.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.743+0000 7f0daf6af640 1 -- start start 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0daf6af640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f0da815d8a0 0x7f0da806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0daf6af640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0da815eb70 0x7f0da80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0daf6af640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da80cca00 0x7f0da815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0daf6af640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da806bf30 con 0x7f0da815eb70 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0daf6af640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da80cb9e0 con 0x7f0da815d8a0 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0daf6af640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80cbcc0 con 0x7f0da80cca00 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0da815eb70 0x7f0da80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0da815eb70 0x7f0da80cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48260/0 (socket says 172.21.15.31:48260) 2024-01-19T04:55:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 -- 172.21.15.31:0/2762830586 learned_addr learned my addr 172.21.15.31:0/2762830586 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dad44b640 1 --2- 172.21.15.31:0/2762830586 >> v2:172.21.15.137:3300/0 conn(0x7f0da815d8a0 0x7f0da806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dadc4c640 1 --2- 172.21.15.31:0/2762830586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da80cca00 0x7f0da815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 -- 172.21.15.31:0/2762830586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da80cca00 msgr2=0x7f0da815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 --2- 172.21.15.31:0/2762830586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da80cca00 0x7f0da815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 -- 172.21.15.31:0/2762830586 >> v2:172.21.15.137:3300/0 conn(0x7f0da815d8a0 msgr2=0x7f0da806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 --2- 172.21.15.31:0/2762830586 >> v2:172.21.15.137:3300/0 conn(0x7f0da815d8a0 0x7f0da806b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dacc4a640 1 -- 172.21.15.31:0/2762830586 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80cbfa0 con 0x7f0da815eb70 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.744+0000 7f0dad44b640 1 --2- 172.21.15.31:0/2762830586 >> v2:172.21.15.137:3300/0 conn(0x7f0da815d8a0 0x7f0da806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.745+0000 7f0dacc4a640 1 --2- 172.21.15.31:0/2762830586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815eb70 0x7f0da80cc4c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0da80d22d0 tx=0x7f0d9802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db7765b78996915 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.745+0000 7f0d9ffff640 1 -- 172.21.15.31:0/2762830586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d980317d0 con 0x7f0da815eb70 2024-01-19T04:55:50.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.745+0000 7f0d9ffff640 1 -- 172.21.15.31:0/2762830586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98031930 con 0x7f0da815eb70 2024-01-19T04:55:50.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.745+0000 7f0d9ffff640 1 -- 172.21.15.31:0/2762830586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98031cc0 con 0x7f0da815eb70 2024-01-19T04:55:50.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.745+0000 7f0daf6af640 1 -- 172.21.15.31:0/2762830586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815eb70 msgr2=0x7f0da80cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 --2- 172.21.15.31:0/2762830586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815eb70 0x7f0da80cc4c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0da80d22d0 tx=0x7f0d9802b870 comp rx=0 tx=0).stop 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 -- 172.21.15.31:0/2762830586 shutdown_connections 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 --2- 172.21.15.31:0/2762830586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da80cca00 0x7f0da815f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 --2- 172.21.15.31:0/2762830586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815eb70 0x7f0da80cc4c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 --2- 172.21.15.31:0/2762830586 >> v2:172.21.15.137:3300/0 conn(0x7f0da815d8a0 0x7f0da806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 -- 172.21.15.31:0/2762830586 >> 172.21.15.31:0/2762830586 conn(0x7f0da806abe0 msgr2=0x7f0da8066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 -- 172.21.15.31:0/2762830586 shutdown_connections 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.746+0000 7f0daf6af640 1 -- 172.21.15.31:0/2762830586 wait complete. 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.747+0000 7f0daf6af640 1 Processor -- start 2024-01-19T04:55:50.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.747+0000 7f0daf6af640 1 -- start start 2024-01-19T04:55:50.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0daf6af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da80cca00 0x7f0da8150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0daf6af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815d8a0 0x7f0da80c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0daf6af640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da815eb70 0x7f0da80c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0daf6af640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80ca9f0 con 0x7f0da815d8a0 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0daf6af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80b3e80 con 0x7f0da80cca00 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0daf6af640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80b4160 con 0x7f0da815eb70 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815d8a0 0x7f0da80c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815d8a0 0x7f0da80c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48272/0 (socket says 172.21.15.31:48272) 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 -- 172.21.15.31:0/3374105963 learned_addr learned my addr 172.21.15.31:0/3374105963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dad44b640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da80cca00 0x7f0da8150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dadc4c640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da815eb70 0x7f0da80c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 -- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da815eb70 msgr2=0x7f0da80c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da815eb70 0x7f0da80c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 -- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da80cca00 msgr2=0x7f0da8150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:50.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da80cca00 0x7f0da8150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 -- 172.21.15.31:0/3374105963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980094d0 con 0x7f0da815d8a0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0dacc4a640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815d8a0 0x7f0da80c5450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0d9dffb640 1 -- 172.21.15.31:0/3374105963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98032870 con 0x7f0da815d8a0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0d9dffb640 1 -- 172.21.15.31:0/3374105963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98032a10 con 0x7f0da815d8a0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0d9dffb640 1 -- 172.21.15.31:0/3374105963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98032dc0 con 0x7f0da815d8a0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.748+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da80b4440 con 0x7f0da815d8a0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.749+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da80ca610 con 0x7f0da815d8a0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.750+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da80696e0 con 0x7f0da815d8a0 2024-01-19T04:55:50.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.750+0000 7f0d9dffb640 1 -- 172.21.15.31:0/3374105963 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0d98030070 con 0x7f0da815d8a0 2024-01-19T04:55:50.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.750+0000 7f0d9dffb640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0d8c06aed0 0x7f0d8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:50.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.750+0000 7f0d9dffb640 1 -- 172.21.15.31:0/3374105963 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d980ab380 con 0x7f0da815d8a0 2024-01-19T04:55:50.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.751+0000 7f0dad44b640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0d8c06aed0 0x7f0d8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:50.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.751+0000 7f0dad44b640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0d8c06aed0 0x7f0d8c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0da400ab60 tx=0x7f0da4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:50.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:50.755+0000 7f0d9dffb640 1 -- 172.21.15.31:0/3374105963 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0d98076300 con 0x7f0da815d8a0 2024-01-19T04:55:51.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.000+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f0da80cab30 con 0x7f0da815d8a0 2024-01-19T04:55:51.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.538+0000 7f0d9dffb640 1 -- 172.21.15.31:0/3374105963 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v50) v1 ==== 150+0+0 (secure 0 0 0) 0x7f0d98079920 con 0x7f0da815d8a0 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0d8c06aed0 msgr2=0x7f0d8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0d8c06aed0 0x7f0d8c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0da400ab60 tx=0x7f0da4008040 comp rx=0 tx=0).stop 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815d8a0 msgr2=0x7f0da80c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815d8a0 0x7f0da80c5450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d980046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 shutdown_connections 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0d8c06aed0 0x7f0d8c06d390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0da815eb70 0x7f0da80c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0da815d8a0 0x7f0da80c5450 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 --2- 172.21.15.31:0/3374105963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da80cca00 0x7f0da8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 >> 172.21.15.31:0/3374105963 conn(0x7f0da806abe0 msgr2=0x7f0da80b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:51.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.540+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 shutdown_connections 2024-01-19T04:55:51.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.541+0000 7f0daf6af640 1 -- 172.21.15.31:0/3374105963 wait complete. 2024-01-19T04:55:51.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-19T04:55:51.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.660+0000 7fc73da46640 1 Processor -- start 2024-01-19T04:55:51.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.660+0000 7fc73da46640 1 -- start start 2024-01-19T04:55:51.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc73da46640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7380d11b0 0x7fc7380d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:51.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc73da46640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7380d1fe0 0x7fc7380b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc73da46640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7380cef10 con 0x7fc73815d700 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc73da46640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7380ce300 con 0x7fc7380d1fe0 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc73da46640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380ce5e0 con 0x7fc7380d11b0 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7377fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7380d1fe0 0x7fc7380b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7377fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7380d1fe0 0x7fc7380b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48284/0 (socket says 172.21.15.31:48284) 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7377fe640 1 -- 172.21.15.31:0/2884040717 learned_addr learned my addr 172.21.15.31:0/2884040717 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7367fc640 1 --2- 172.21.15.31:0/2884040717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7380d11b0 0x7fc7380d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7377fe640 1 -- 172.21.15.31:0/2884040717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7380d11b0 msgr2=0x7fc7380d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7377fe640 1 --2- 172.21.15.31:0/2884040717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7380d11b0 0x7fc7380d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7377fe640 1 -- 172.21.15.31:0/2884040717 >> v1:172.21.15.137:6789/0 conn(0x7fc73815d700 legacy=0x7fc7380d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7377fe640 1 -- 172.21.15.31:0/2884040717 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7380ce8c0 con 0x7fc7380d1fe0 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.661+0000 7fc7367fc640 1 --2- 172.21.15.31:0/2884040717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7380d11b0 0x7fc7380d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.662+0000 7fc7377fe640 1 --2- 172.21.15.31:0/2884040717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d1fe0 0x7fc7380b2890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc728009e50 tx=0x7fc72802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9267f8f386a1917 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.662+0000 7fc735ffb640 1 -- 172.21.15.31:0/2884040717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72802cb00 con 0x7fc7380d1fe0 2024-01-19T04:55:51.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.662+0000 7fc735ffb640 1 -- 172.21.15.31:0/2884040717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72802cc60 con 0x7fc7380d1fe0 2024-01-19T04:55:51.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.662+0000 7fc735ffb640 1 -- 172.21.15.31:0/2884040717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7280343f0 con 0x7fc7380d1fe0 2024-01-19T04:55:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 -- 172.21.15.31:0/2884040717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d1fe0 msgr2=0x7fc7380b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 --2- 172.21.15.31:0/2884040717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d1fe0 0x7fc7380b2890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc728009e50 tx=0x7fc72802bf60 comp rx=0 tx=0).stop 2024-01-19T04:55:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 -- 172.21.15.31:0/2884040717 shutdown_connections 2024-01-19T04:55:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 --2- 172.21.15.31:0/2884040717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d1fe0 0x7fc7380b2890 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 --2- 172.21.15.31:0/2884040717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7380d11b0 0x7fc7380d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 -- 172.21.15.31:0/2884040717 >> 172.21.15.31:0/2884040717 conn(0x7fc7380d32f0 msgr2=0x7fc7380d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 -- 172.21.15.31:0/2884040717 shutdown_connections 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.663+0000 7fc73da46640 1 -- 172.21.15.31:0/2884040717 wait complete. 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.664+0000 7fc73da46640 1 Processor -- start 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.664+0000 7fc73da46640 1 -- start start 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc73da46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d11b0 0x7fc73814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc73da46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7380d1fe0 0x7fc7381f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc73da46640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc73815d700 0x7fc7381f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc73da46640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380c9cb0 con 0x7fc7380d11b0 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc73da46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380c99d0 con 0x7fc7380d1fe0 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc73da46640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380b0b40 con 0x7fc73815d700 2024-01-19T04:55:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d11b0 0x7fc73814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d11b0 0x7fc73814c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48300/0 (socket says 172.21.15.31:48300) 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 -- 172.21.15.31:0/3545347508 learned_addr learned my addr 172.21.15.31:0/3545347508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc7367fc640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7380d1fe0 0x7fc7381f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc7377fe640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc73815d700 0x7fc7381f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 -- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc73815d700 msgr2=0x7fc7381f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc73815d700 0x7fc7381f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 -- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7380d1fe0 msgr2=0x7fc7381f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7380d1fe0 0x7fc7381f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 -- 172.21.15.31:0/3545347508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728009b30 con 0x7fc7380d11b0 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc7377fe640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc73815d700 0x7fc7381f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc736ffd640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d11b0 0x7fc73814c210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc73000b3f0 tx=0x7fc73000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc72ffff640 1 -- 172.21.15.31:0/3545347508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011830 con 0x7fc7380d11b0 2024-01-19T04:55:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc72ffff640 1 -- 172.21.15.31:0/3545347508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc730011990 con 0x7fc7380d11b0 2024-01-19T04:55:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7380b0e20 con 0x7fc7380d11b0 2024-01-19T04:55:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.665+0000 7fc72ffff640 1 -- 172.21.15.31:0/3545347508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011d40 con 0x7fc7380d11b0 2024-01-19T04:55:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.666+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73814cc40 con 0x7fc7380d11b0 2024-01-19T04:55:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.667+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7380b1340 con 0x7fc7380d11b0 2024-01-19T04:55:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.667+0000 7fc72ffff640 1 -- 172.21.15.31:0/3545347508 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc73001a400 con 0x7fc7380d11b0 2024-01-19T04:55:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.667+0000 7fc72ffff640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71806aed0 0x7fc71806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.668+0000 7fc72ffff640 1 -- 172.21.15.31:0/3545347508 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7300893a0 con 0x7fc7380d11b0 2024-01-19T04:55:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.668+0000 7fc7367fc640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71806aed0 0x7fc71806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.668+0000 7fc7367fc640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71806aed0 0x7fc71806d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc72000ab60 tx=0x7fc720008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.672+0000 7fc72ffff640 1 -- 172.21.15.31:0/3545347508 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc7380b1340 con 0x7fc7380d11b0 2024-01-19T04:55:51.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.849+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc738162190 con 0x7fc7380d11b0 2024-01-19T04:55:51.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.852+0000 7fc72ffff640 1 -- 172.21.15.31:0/3545347508 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc730054300 con 0x7fc7380d11b0 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71806aed0 msgr2=0x7fc71806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71806aed0 0x7fc71806d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc72000ab60 tx=0x7fc720008040 comp rx=0 tx=0).stop 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d11b0 msgr2=0x7fc73814c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d11b0 0x7fc73814c210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc73000b3f0 tx=0x7fc73000b8b0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 shutdown_connections 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc71806aed0 0x7fc71806d390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc73815d700 0x7fc7381f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7380d1fe0 0x7fc7381f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 --2- 172.21.15.31:0/3545347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7380d11b0 0x7fc73814c210 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:51.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 >> 172.21.15.31:0/3545347508 conn(0x7fc7380d32f0 msgr2=0x7fc73815b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:51.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 shutdown_connections 2024-01-19T04:55:51.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:51.854+0000 7fc73da46640 1 -- 172.21.15.31:0/3545347508 wait complete. 2024-01-19T04:55:51.864 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-19T04:55:51.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:55:51.910 DEBUG:teuthology.orchestra.run.smithi137:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:55:51.963 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-19T04:55:51.963 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:55:51.963 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:51.978 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:55:51.979 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:52.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:52.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.175+0000 7fa8ccb5a640 1 Processor -- start 2024-01-19T04:55:52.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.175+0000 7fa8ccb5a640 1 -- start start 2024-01-19T04:55:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8ccb5a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c80cc230 0x7fa8c80b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8ccb5a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8c8142320 0x7fa8c80b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8ccb5a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c8142890 con 0x7fa8c80c4db0 2024-01-19T04:55:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8ccb5a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c8142b70 con 0x7fa8c8142320 2024-01-19T04:55:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8ccb5a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8142e50 con 0x7fa8c80cc230 2024-01-19T04:55:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c5d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8c8142320 0x7fa8c80b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c6d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa8c80c4db0 0x7fa8c8143930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54974/0 (socket says 172.21.15.31:54974) 2024-01-19T04:55:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c6d9d640 1 -- 172.21.15.31:0/4203301336 learned_addr learned my addr 172.21.15.31:0/4203301336 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c5d9b640 1 -- 172.21.15.31:0/4203301336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c80cc230 msgr2=0x7fa8c80b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c5d9b640 1 --2- 172.21.15.31:0/4203301336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c80cc230 0x7fa8c80b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c5d9b640 1 -- 172.21.15.31:0/4203301336 >> v1:172.21.15.31:6789/0 conn(0x7fa8c80c4db0 legacy=0x7fa8c8143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c559a640 1 -- 172.21.15.31:0/4203301336 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573429471 0 0) 0x7fa8c8142890 con 0x7fa8c80c4db0 2024-01-19T04:55:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.176+0000 7fa8c5d9b640 1 -- 172.21.15.31:0/4203301336 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8143130 con 0x7fa8c8142320 2024-01-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.177+0000 7fa8c5d9b640 1 --2- 172.21.15.31:0/4203301336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c8142320 0x7fa8c80b3280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00097a0 tx=0x7fa8b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b404f2909199ced server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.177+0000 7fa8c559a640 1 -- 172.21.15.31:0/4203301336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0031830 con 0x7fa8c8142320 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.177+0000 7fa8c559a640 1 -- 172.21.15.31:0/4203301336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b0031990 con 0x7fa8c8142320 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.177+0000 7fa8c559a640 1 -- 172.21.15.31:0/4203301336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0031d20 con 0x7fa8c8142320 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/4203301336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c8142320 msgr2=0x7fa8c80b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/4203301336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c8142320 0x7fa8c80b3280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00097a0 tx=0x7fa8b002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/4203301336 shutdown_connections 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/4203301336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c8142320 0x7fa8c80b3280 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/4203301336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c80cc230 0x7fa8c80b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/4203301336 >> 172.21.15.31:0/4203301336 conn(0x7fa8c806abe0 msgr2=0x7fa8c8066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/4203301336 shutdown_connections 2024-01-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.178+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/4203301336 wait complete. 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.179+0000 7fa8ccb5a640 1 Processor -- start 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.179+0000 7fa8ccb5a640 1 -- start start 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8ccb5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c80c4db0 0x7fa8c80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8ccb5a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c80cc230 0x7fa8c81f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8ccb5a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c8142320 0x7fa8c80b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8ccb5a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80b53a0 con 0x7fa8c80cc230 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8ccb5a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80d0c10 con 0x7fa8c80c4db0 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8ccb5a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80d02e0 con 0x7fa8c8142320 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c80cc230 0x7fa8c81f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c80cc230 0x7fa8c81f0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48326/0 (socket says 172.21.15.31:48326) 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 -- 172.21.15.31:0/437721056 learned_addr learned my addr 172.21.15.31:0/437721056 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c6d9d640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c8142320 0x7fa8c80b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c659c640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c80c4db0 0x7fa8c80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 -- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c8142320 msgr2=0x7fa8c80b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c8142320 0x7fa8c80b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 -- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c80c4db0 msgr2=0x7fa8c80b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c80c4db0 0x7fa8c80b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c5d9b640 1 -- 172.21.15.31:0/437721056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b0009400 con 0x7fa8c80cc230 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.180+0000 7fa8c659c640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c80c4db0 0x7fa8c80b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.181+0000 7fa8c5d9b640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c80cc230 0x7fa8c81f0df0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0034040 tx=0x7fa8b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.181+0000 7fa8beffd640 1 -- 172.21.15.31:0/437721056 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00317d0 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.181+0000 7fa8beffd640 1 -- 172.21.15.31:0/437721056 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b0004030 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.181+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c80d05c0 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.181+0000 7fa8beffd640 1 -- 172.21.15.31:0/437721056 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00043e0 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.181+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c80c84f0 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.182+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c80696e0 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.182+0000 7fa8beffd640 1 -- 172.21.15.31:0/437721056 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa8b0043070 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.182+0000 7fa8beffd640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa8a806aed0 0x7fa8a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.183+0000 7fa8beffd640 1 -- 172.21.15.31:0/437721056 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8b00ab3a0 con 0x7fa8c80cc230 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.183+0000 7fa8c659c640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa8a806aed0 0x7fa8a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.183+0000 7fa8c659c640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa8a806aed0 0x7fa8a806d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0006fd0 tx=0x7fa8c0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.187+0000 7fa8beffd640 1 -- 172.21.15.31:0/437721056 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa8b0076260 con 0x7fa8c80cc230 2024-01-19T04:55:52.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.365+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c80ce630 con 0x7fa8c80cc230 2024-01-19T04:55:52.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.366+0000 7fa8beffd640 1 -- 172.21.15.31:0/437721056 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fa8b0079880 con 0x7fa8c80cc230 2024-01-19T04:55:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:52.366 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa8a806aed0 msgr2=0x7fa8a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa8a806aed0 0x7fa8a806d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0006fd0 tx=0x7fa8c0008040 comp rx=0 tx=0).stop 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c80cc230 msgr2=0x7fa8c81f0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c80cc230 0x7fa8c81f0df0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0034040 tx=0x7fa8b00046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 shutdown_connections 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa8a806aed0 0x7fa8a806d390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa8c8142320 0x7fa8c80b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c80cc230 0x7fa8c81f0df0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 --2- 172.21.15.31:0/437721056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c80c4db0 0x7fa8c80b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 >> 172.21.15.31:0/437721056 conn(0x7fa8c806abe0 msgr2=0x7fa8c80b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:52.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 shutdown_connections 2024-01-19T04:55:52.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.368+0000 7fa8ccb5a640 1 -- 172.21.15.31:0/437721056 wait complete. 2024-01-19T04:55:52.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:52.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.485+0000 7f01f87f1640 1 Processor -- start 2024-01-19T04:55:52.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.485+0000 7f01f87f1640 1 -- start start 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f87f1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d3a0 0x7f01f006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f87f1640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01f00cf570 0x7f01f0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f87f1640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f00cf9f0 con 0x7f01f015d780 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f87f1640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00cfcd0 con 0x7f01f00cf570 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f87f1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00cffb0 con 0x7f01f015d3a0 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f658d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d3a0 0x7f01f006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f5d8c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01f00cf570 0x7f01f0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f6d8e640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f01f015d780 0x7f01f00cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:41280/0 (socket says 172.21.15.31:41280) 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f5d8c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01f00cf570 0x7f01f0161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:41316/0 (socket says 172.21.15.31:41316) 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f6d8e640 1 -- 172.21.15.31:0/1051685953 learned_addr learned my addr 172.21.15.31:0/1051685953 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f5d8c640 1 -- 172.21.15.31:0/1051685953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d3a0 msgr2=0x7f01f006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:52.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f5d8c640 1 --2- 172.21.15.31:0/1051685953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d3a0 0x7f01f006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f5d8c640 1 -- 172.21.15.31:0/1051685953 >> v1:172.21.15.137:6789/0 conn(0x7f01f015d780 legacy=0x7f01f00cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f5d8c640 1 -- 172.21.15.31:0/1051685953 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00d0290 con 0x7f01f00cf570 2024-01-19T04:55:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.486+0000 7f01f558b640 1 -- 172.21.15.31:0/1051685953 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972690376 0 0) 0x7f01f00cf9f0 con 0x7f01f015d780 2024-01-19T04:55:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.487+0000 7f01f5d8c640 1 --2- 172.21.15.31:0/1051685953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 0x7f01f0161150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01e00097a0 tx=0x7f01e002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71c31bff39bb934b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.487+0000 7f01f558b640 1 -- 172.21.15.31:0/1051685953 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0031830 con 0x7f01f00cf570 2024-01-19T04:55:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.487+0000 7f01f558b640 1 -- 172.21.15.31:0/1051685953 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0031990 con 0x7f01f00cf570 2024-01-19T04:55:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.487+0000 7f01f558b640 1 -- 172.21.15.31:0/1051685953 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0031d20 con 0x7f01f00cf570 2024-01-19T04:55:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 -- 172.21.15.31:0/1051685953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 msgr2=0x7f01f0161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 --2- 172.21.15.31:0/1051685953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 0x7f01f0161150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01e00097a0 tx=0x7f01e002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 -- 172.21.15.31:0/1051685953 shutdown_connections 2024-01-19T04:55:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 --2- 172.21.15.31:0/1051685953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 0x7f01f0161150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 --2- 172.21.15.31:0/1051685953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d3a0 0x7f01f006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 -- 172.21.15.31:0/1051685953 >> 172.21.15.31:0/1051685953 conn(0x7f01f00d1100 msgr2=0x7f01f00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 -- 172.21.15.31:0/1051685953 shutdown_connections 2024-01-19T04:55:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.488+0000 7f01f87f1640 1 -- 172.21.15.31:0/1051685953 wait complete. 2024-01-19T04:55:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.489+0000 7f01f87f1640 1 Processor -- start 2024-01-19T04:55:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.489+0000 7f01f87f1640 1 -- start start 2024-01-19T04:55:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f87f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 0x7f01f009b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f87f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01f015d3a0 0x7f01f009ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f87f1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d780 0x7f01f00aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f87f1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0147280 con 0x7f01f015d3a0 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f87f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0147560 con 0x7f01f00cf570 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f87f1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0147840 con 0x7f01f015d780 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01f015d3a0 0x7f01f009ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01f015d3a0 0x7f01f009ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48342/0 (socket says 172.21.15.31:48342) 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 -- 172.21.15.31:0/2309884478 learned_addr learned my addr 172.21.15.31:0/2309884478 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f658d640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 0x7f01f009b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f6d8e640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d780 0x7f01f00aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 -- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d780 msgr2=0x7f01f00aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d780 0x7f01f00aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 -- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 msgr2=0x7f01f009b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 0x7f01f009b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.490+0000 7f01f5d8c640 1 -- 172.21.15.31:0/2309884478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0009400 con 0x7f01f015d3a0 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.491+0000 7f01f5d8c640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01f015d3a0 0x7f01f009ccb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f01e002bee0 tx=0x7f01e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.491+0000 7f01e6ffd640 1 -- 172.21.15.31:0/2309884478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e003a6f0 con 0x7f01f015d3a0 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.491+0000 7f01e6ffd640 1 -- 172.21.15.31:0/2309884478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e003a870 con 0x7f01f015d3a0 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.491+0000 7f01e6ffd640 1 -- 172.21.15.31:0/2309884478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e003ac20 con 0x7f01f015d3a0 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.491+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f0147b20 con 0x7f01f015d3a0 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.491+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01480e0 con 0x7f01f015d3a0 2024-01-19T04:55:52.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.492+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f0148270 con 0x7f01f015d3a0 2024-01-19T04:55:52.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.492+0000 7f01e6ffd640 1 -- 172.21.15.31:0/2309884478 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f01e0030380 con 0x7f01f015d3a0 2024-01-19T04:55:52.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.493+0000 7f01e6ffd640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01d406aed0 0x7f01d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.493+0000 7f01e6ffd640 1 -- 172.21.15.31:0/2309884478 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01e00aaf60 con 0x7f01f015d3a0 2024-01-19T04:55:52.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.493+0000 7f01f658d640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01d406aed0 0x7f01d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.493+0000 7f01f658d640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01d406aed0 0x7f01d406d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009ac0 tx=0x7f01ec008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.497+0000 7f01e6ffd640 1 -- 172.21.15.31:0/2309884478 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f01e0075e30 con 0x7f01f015d3a0 2024-01-19T04:55:52.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.678+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01f0074900 con 0x7f01f015d3a0 2024-01-19T04:55:52.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.678+0000 7f01e6ffd640 1 -- 172.21.15.31:0/2309884478 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01e00350a0 con 0x7f01f015d3a0 2024-01-19T04:55:52.678 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01d406aed0 msgr2=0x7f01d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01d406aed0 0x7f01d406d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009ac0 tx=0x7f01ec008040 comp rx=0 tx=0).stop 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01f015d3a0 msgr2=0x7f01f009ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01f015d3a0 0x7f01f009ccb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f01e002bee0 tx=0x7f01e00046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 shutdown_connections 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01d406aed0 0x7f01d406d390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01f015d780 0x7f01f00aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01f015d3a0 0x7f01f009ccb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 --2- 172.21.15.31:0/2309884478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f00cf570 0x7f01f009b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 >> 172.21.15.31:0/2309884478 conn(0x7f01f00d1100 msgr2=0x7f01f015e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 shutdown_connections 2024-01-19T04:55:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.680+0000 7f01f87f1640 1 -- 172.21.15.31:0/2309884478 wait complete. 2024-01-19T04:55:52.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.795+0000 7f3059807640 1 Processor -- start 2024-01-19T04:55:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.795+0000 7f3059807640 1 -- start start 2024-01-19T04:55:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3059807640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f305406f900 0x7f3054071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3059807640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406a5b0 0x7f30540cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3059807640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305415d9d0 con 0x7f30540cc1e0 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3059807640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305406fce0 con 0x7f305406f900 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3059807640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305406aa40 con 0x7f305406a5b0 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f30527fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406a5b0 0x7f30540cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3052ffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f305406f900 0x7f3054071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3052ffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f305406f900 0x7f3054071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:41322/0 (socket says 172.21.15.31:41322) 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f30527fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406a5b0 0x7f30540cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:46274/0 (socket says 172.21.15.31:46274) 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f3052ffd640 1 -- 172.21.15.31:0/3945131189 learned_addr learned my addr 172.21.15.31:0/3945131189 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.796+0000 7f30537fe640 1 --1- 172.21.15.31:0/3945131189 >> v1:172.21.15.137:6789/0 conn(0x7f30540cc1e0 0x7f30541610c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:41292/0 (socket says 172.21.15.31:41292) 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.797+0000 7f3052ffd640 1 -- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406a5b0 msgr2=0x7f30540cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.797+0000 7f3052ffd640 1 --2- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406a5b0 0x7f30540cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.797+0000 7f3052ffd640 1 -- 172.21.15.31:0/3945131189 >> v1:172.21.15.137:6789/0 conn(0x7f30540cc1e0 legacy=0x7f30541610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.797+0000 7f3052ffd640 1 -- 172.21.15.31:0/3945131189 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305406ad20 con 0x7f305406f900 2024-01-19T04:55:52.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.797+0000 7f30527fc640 1 --2- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406a5b0 0x7f30540cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3052ffd640 1 --2- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f305406f900 0x7f3054071e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3044009e50 tx=0x7f304402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e2815b85ac96ab server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3051ffb640 1 -- 172.21.15.31:0/3945131189 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304402cb00 con 0x7f305406f900 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3051ffb640 1 -- 172.21.15.31:0/3945131189 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304402cc60 con 0x7f305406f900 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3051ffb640 1 -- 172.21.15.31:0/3945131189 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30440343f0 con 0x7f305406f900 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 -- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f305406f900 msgr2=0x7f3054071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 --2- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f305406f900 0x7f3054071e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3044009e50 tx=0x7f304402bf60 comp rx=0 tx=0).stop 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 -- 172.21.15.31:0/3945131189 shutdown_connections 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 --2- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406a5b0 0x7f30540cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 --2- 172.21.15.31:0/3945131189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f305406f900 0x7f3054071e40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 -- 172.21.15.31:0/3945131189 >> 172.21.15.31:0/3945131189 conn(0x7f30540d0000 msgr2=0x7f30540d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 -- 172.21.15.31:0/3945131189 shutdown_connections 2024-01-19T04:55:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.798+0000 7f3059807640 1 -- 172.21.15.31:0/3945131189 wait complete. 2024-01-19T04:55:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.799+0000 7f3059807640 1 Processor -- start 2024-01-19T04:55:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f3059807640 1 -- start start 2024-01-19T04:55:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f3059807640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406f900 0x7f3054158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f3059807640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540cc1e0 0x7f305415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f3059807640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30540b3d80 0x7f3054071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f3059807640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540b38f0 con 0x7f30540cc1e0 2024-01-19T04:55:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f3059807640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540b2d20 con 0x7f30540b3d80 2024-01-19T04:55:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f3059807640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30540b2fc0 con 0x7f305406f900 2024-01-19T04:55:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f30527fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540cc1e0 0x7f305415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f30527fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540cc1e0 0x7f305415f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48348/0 (socket says 172.21.15.31:48348) 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.800+0000 7f30527fc640 1 -- 172.21.15.31:0/1786576896 learned_addr learned my addr 172.21.15.31:0/1786576896 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30537fe640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30540b3d80 0x7f3054071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f3052ffd640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406f900 0x7f3054158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30527fc640 1 -- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406f900 msgr2=0x7f3054158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30527fc640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406f900 0x7f3054158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30527fc640 1 -- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30540b3d80 msgr2=0x7f3054071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30527fc640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30540b3d80 0x7f3054071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30527fc640 1 -- 172.21.15.31:0/1786576896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c009400 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30537fe640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30540b3d80 0x7f3054071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f30527fc640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540cc1e0 0x7f305415f960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f303c00e960 tx=0x7f303c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f304bfff640 1 -- 172.21.15.31:0/1786576896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00ca00 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f304bfff640 1 -- 172.21.15.31:0/1786576896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c00cb60 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3044009b30 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f304bfff640 1 -- 172.21.15.31:0/1786576896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c013470 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.801+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30540b3580 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.802+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3054074900 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.802+0000 7f304bfff640 1 -- 172.21.15.31:0/1786576896 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f303c0135d0 con 0x7f30540cc1e0 2024-01-19T04:55:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.803+0000 7f304bfff640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f303406aed0 0x7f303406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.803+0000 7f304bfff640 1 -- 172.21.15.31:0/1786576896 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f303c089860 con 0x7f30540cc1e0 2024-01-19T04:55:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.803+0000 7f3052ffd640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f303406aed0 0x7f303406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.804+0000 7f3052ffd640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f303406aed0 0x7f303406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3044035000 tx=0x7f3044030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.807+0000 7f304bfff640 1 -- 172.21.15.31:0/1786576896 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f303c0547c0 con 0x7f30540cc1e0 2024-01-19T04:55:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.985+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30540b5f80 con 0x7f30540cc1e0 2024-01-19T04:55:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.986+0000 7f304bfff640 1 -- 172.21.15.31:0/1786576896 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f303c057de0 con 0x7f30540cc1e0 2024-01-19T04:55:52.985 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f303406aed0 msgr2=0x7f303406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f303406aed0 0x7f303406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3044035000 tx=0x7f3044030000 comp rx=0 tx=0).stop 2024-01-19T04:55:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540cc1e0 msgr2=0x7f305415f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540cc1e0 0x7f305415f960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f303c00e960 tx=0x7f303c00ee20 comp rx=0 tx=0).stop 2024-01-19T04:55:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 shutdown_connections 2024-01-19T04:55:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f303406aed0 0x7f303406d390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30540b3d80 0x7f3054071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540cc1e0 0x7f305415f960 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 --2- 172.21.15.31:0/1786576896 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f305406f900 0x7f3054158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.987+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 >> 172.21.15.31:0/1786576896 conn(0x7f30540d0000 msgr2=0x7f30540afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.988+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 shutdown_connections 2024-01-19T04:55:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:52.988+0000 7f3059807640 1 -- 172.21.15.31:0/1786576896 wait complete. 2024-01-19T04:55:52.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.103+0000 7f4c20ada640 1 Processor -- start 2024-01-19T04:55:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.103+0000 7f4c20ada640 1 -- start start 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.104+0000 7f4c20ada640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4c1c0d1c20 0x7f4c1c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.104+0000 7f4c20ada640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c0cf160 con 0x7f4c1c0d3ea0 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.104+0000 7f4c20ada640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c0ce8c0 con 0x7f4c1c15d720 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.104+0000 7f4c20ada640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0ceba0 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c1ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4c1c0d3ea0 0x7f4c1c0b3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54982/0 (socket says 172.21.15.31:54982) 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c1ad9d640 1 -- 172.21.15.31:0/3204796791 learned_addr learned my addr 172.21.15.31:0/3204796791 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c19d9b640 1 --2- 172.21.15.31:0/3204796791 >> v2:172.21.15.137:3300/0 conn(0x7f4c1c0d1c20 0x7f4c1c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c1959a640 1 -- 172.21.15.31:0/3204796791 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311243915 0 0) 0x7f4c1c0cf160 con 0x7f4c1c0d3ea0 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c1959a640 1 -- 172.21.15.31:0/3204796791 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c08003520 con 0x7f4c1c0d3ea0 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c1959a640 1 -- 172.21.15.31:0/3204796791 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721187375 0 0) 0x7f4c1c0ce8c0 con 0x7f4c1c15d720 2024-01-19T04:55:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c1959a640 1 -- 172.21.15.31:0/3204796791 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c1c0cf160 con 0x7f4c1c15d720 2024-01-19T04:55:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c19d9b640 1 -- 172.21.15.31:0/3204796791 >> v1:172.21.15.137:6789/0 conn(0x7f4c1c15d720 legacy=0x7f4c1c0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c19d9b640 1 -- 172.21.15.31:0/3204796791 >> v1:172.21.15.31:6789/0 conn(0x7f4c1c0d3ea0 legacy=0x7f4c1c0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.105+0000 7f4c19d9b640 1 -- 172.21.15.31:0/3204796791 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0cee80 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.106+0000 7f4c19d9b640 1 --2- 172.21.15.31:0/3204796791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c0d0ee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c040097a0 tx=0x7f4c0402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f29bc675d7ad4a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.106+0000 7f4c1959a640 1 -- 172.21.15.31:0/3204796791 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0402e070 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.106+0000 7f4c1959a640 1 -- 172.21.15.31:0/3204796791 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c040315a0 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.106+0000 7f4c1959a640 1 -- 172.21.15.31:0/3204796791 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04031910 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.106+0000 7f4c20ada640 1 -- 172.21.15.31:0/3204796791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d1c20 msgr2=0x7f4c1c0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.106+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3204796791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c0d0ee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c040097a0 tx=0x7f4c0402b840 comp rx=0 tx=0).stop 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.107+0000 7f4c20ada640 1 -- 172.21.15.31:0/3204796791 shutdown_connections 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.107+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3204796791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c0d0ee0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.107+0000 7f4c20ada640 1 -- 172.21.15.31:0/3204796791 >> 172.21.15.31:0/3204796791 conn(0x7f4c1c0d3310 msgr2=0x7f4c1c0d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.107+0000 7f4c20ada640 1 -- 172.21.15.31:0/3204796791 shutdown_connections 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.107+0000 7f4c20ada640 1 -- 172.21.15.31:0/3204796791 wait complete. 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.108+0000 7f4c20ada640 1 Processor -- start 2024-01-19T04:55:53.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.108+0000 7f4c20ada640 1 -- start start 2024-01-19T04:55:53.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c1477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d3ea0 0x7f4c1c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c1c15d720 0x7f4c1c15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c8490 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c160f90 con 0x7f4c1c0d3ea0 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1f3420 con 0x7f4c1c15d720 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c1477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c1477a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48358/0 (socket says 172.21.15.31:48358) 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 -- 172.21.15.31:0/3315556505 learned_addr learned my addr 172.21.15.31:0/3315556505 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1ad9d640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c1c15d720 0x7f4c1c15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c19d9b640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d3ea0 0x7f4c1c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 -- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c1c15d720 msgr2=0x7f4c1c15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c1c15d720 0x7f4c1c15e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 -- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d3ea0 msgr2=0x7f4c1c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d3ea0 0x7f4c1c15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 -- 172.21.15.31:0/3315556505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04009400 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c19d9b640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d3ea0 0x7f4c1c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c1a59c640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c1477a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c7a0 tx=0x7f4c0c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c12ffd640 1 -- 172.21.15.31:0/3315556505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c003b70 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c12ffd640 1 -- 172.21.15.31:0/3315556505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c003cf0 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c0c7af0 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c12ffd640 1 -- 172.21.15.31:0/3315556505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00aa30 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.109+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c0c80b0 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.110+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c074900 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.111+0000 7f4c12ffd640 1 -- 172.21.15.31:0/3315556505 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4c0c00ab90 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.111+0000 7f4c12ffd640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4bfc06aed0 0x7f4bfc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.111+0000 7f4c12ffd640 1 -- 172.21.15.31:0/3315556505 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c0c089410 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.111+0000 7f4c19d9b640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4bfc06aed0 0x7f4bfc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.112+0000 7f4c19d9b640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4bfc06aed0 0x7f4bfc06d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4c04036000 tx=0x7f4c0403d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.116+0000 7f4c12ffd640 1 -- 172.21.15.31:0/3315556505 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4c0c054370 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.294+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c1c0b25e0 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.294+0000 7f4c12ffd640 1 -- 172.21.15.31:0/3315556505 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c0c003e60 con 0x7f4c1c0d1c20 2024-01-19T04:55:53.294 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4bfc06aed0 msgr2=0x7f4bfc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4bfc06aed0 0x7f4bfc06d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4c04036000 tx=0x7f4c0403d040 comp rx=0 tx=0).stop 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c1c0d1c20 msgr2=0x7f4c1c1477a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c1477a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c7a0 tx=0x7f4c0c00cc60 comp rx=0 tx=0).stop 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 shutdown_connections 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4bfc06aed0 0x7f4bfc06d390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c1c15d720 0x7f4c1c15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c1c0d3ea0 0x7f4c1c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 --2- 172.21.15.31:0/3315556505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c1c0d1c20 0x7f4c1c1477a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 >> 172.21.15.31:0/3315556505 conn(0x7f4c1c0d3310 msgr2=0x7f4c1c073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 shutdown_connections 2024-01-19T04:55:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.296+0000 7f4c20ada640 1 -- 172.21.15.31:0/3315556505 wait complete. 2024-01-19T04:55:53.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:55:53.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.409+0000 7fdbcd9a9640 1 Processor -- start 2024-01-19T04:55:53.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.409+0000 7fdbcd9a9640 1 -- start start 2024-01-19T04:55:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbcd9a9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc80ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbcd9a9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc8066690 con 0x7fdbc815da00 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbcd9a9640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80ace00 con 0x7fdbc806bcd0 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbcd9a9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc815df70 con 0x7fdbc815af30 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc67fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdbc815da00 0x7fdbc806b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54994/0 (socket says 172.21.15.31:54994) 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc67fc640 1 -- 172.21.15.31:0/3581906857 learned_addr learned my addr 172.21.15.31:0/3581906857 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc6ffd640 1 --2- 172.21.15.31:0/3581906857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc80ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc5ffb640 1 -- 172.21.15.31:0/3581906857 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000588472 0 0) 0x7fdbc8066690 con 0x7fdbc815da00 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc5ffb640 1 -- 172.21.15.31:0/3581906857 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb4003520 con 0x7fdbc815da00 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc5ffb640 1 -- 172.21.15.31:0/3581906857 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327669684 0 0) 0x7fdbc0001110 con 0x7fdbc806bcd0 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc5ffb640 1 -- 172.21.15.31:0/3581906857 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbc8066690 con 0x7fdbc806bcd0 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc6ffd640 1 -- 172.21.15.31:0/3581906857 >> v1:172.21.15.137:6789/0 conn(0x7fdbc806bcd0 legacy=0x7fdbc80b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc6ffd640 1 -- 172.21.15.31:0/3581906857 >> v1:172.21.15.31:6789/0 conn(0x7fdbc815da00 legacy=0x7fdbc806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.410+0000 7fdbc6ffd640 1 -- 172.21.15.31:0/3581906857 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc815e250 con 0x7fdbc815af30 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.411+0000 7fdbc6ffd640 1 --2- 172.21.15.31:0/3581906857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc80ac8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009dc0 tx=0x7fdbb802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddafc1b504fa4c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.411+0000 7fdbc5ffb640 1 -- 172.21.15.31:0/3581906857 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb802ccd0 con 0x7fdbc815af30 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.411+0000 7fdbc5ffb640 1 -- 172.21.15.31:0/3581906857 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb802ce30 con 0x7fdbc815af30 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.411+0000 7fdbc5ffb640 1 -- 172.21.15.31:0/3581906857 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8034660 con 0x7fdbc815af30 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.412+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/3581906857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 msgr2=0x7fdbc80ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.412+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/3581906857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc80ac8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009dc0 tx=0x7fdbb802bf20 comp rx=0 tx=0).stop 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.412+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/3581906857 shutdown_connections 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.412+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/3581906857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc80ac8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.412+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/3581906857 >> 172.21.15.31:0/3581906857 conn(0x7fdbc8070bf0 msgr2=0x7fdbc8071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.412+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/3581906857 shutdown_connections 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.412+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/3581906857 wait complete. 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.413+0000 7fdbcd9a9640 1 Processor -- start 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.413+0000 7fdbcd9a9640 1 -- start start 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbcd9a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc806bcd0 0x7fdbc814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbcd9a9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbcd9a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc815da00 0x7fdbc8159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbcd9a9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc80cd3e0 con 0x7fdbc815da00 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbcd9a9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc806e9a0 con 0x7fdbc806bcd0 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbcd9a9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc814dda0 con 0x7fdbc815af30 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbc77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc815da00 0x7fdbc8159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbc77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc815da00 0x7fdbc8159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48362/0 (socket says 172.21.15.31:48362) 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbc77fe640 1 -- 172.21.15.31:0/2528896723 learned_addr learned my addr 172.21.15.31:0/2528896723 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbc6ffd640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc806bcd0 0x7fdbc814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbc67fc640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.414+0000 7fdbc77fe640 1 -- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 msgr2=0x7fdbc8159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbc77fe640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc8159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbc77fe640 1 -- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc806bcd0 msgr2=0x7fdbc814d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbc77fe640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc806bcd0 0x7fdbc814d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbc77fe640 1 -- 172.21.15.31:0/2528896723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8009a70 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbc6ffd640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc806bcd0 0x7fdbc814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbc67fc640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc8159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbc77fe640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc815da00 0x7fdbc8159860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c760 tx=0x7fdbc000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbbffff640 1 -- 172.21.15.31:0/2528896723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00050d0 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbbffff640 1 -- 172.21.15.31:0/2528896723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0005230 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbbffff640 1 -- 172.21.15.31:0/2528896723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000a640 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc815b490 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.415+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc80cca40 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.416+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc80cd520 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.416+0000 7fdbbffff640 1 -- 172.21.15.31:0/2528896723 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdbc000a7a0 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.417+0000 7fdbbffff640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdba806aed0 0x7fdba806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.417+0000 7fdbbffff640 1 -- 172.21.15.31:0/2528896723 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbc0088890 con 0x7fdbc815da00 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.417+0000 7fdbc6ffd640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdba806aed0 0x7fdba806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.417+0000 7fdbc6ffd640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdba806aed0 0x7fdba806d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8036000 tx=0x7fdbb8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.421+0000 7fdbbffff640 1 -- 172.21.15.31:0/2528896723 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdbc80cd520 con 0x7fdbc815da00 2024-01-19T04:55:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.610+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdbc80ace00 con 0x7fdbc815da00 2024-01-19T04:55:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.611+0000 7fdbbffff640 1 -- 172.21.15.31:0/2528896723 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdbc80ace00 con 0x7fdbc815da00 2024-01-19T04:55:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.612+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdba806aed0 msgr2=0x7fdba806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.612+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdba806aed0 0x7fdba806d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8036000 tx=0x7fdbb8030000 comp rx=0 tx=0).stop 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.612+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc815da00 msgr2=0x7fdbc8159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.612+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc815da00 0x7fdbc8159860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c760 tx=0x7fdbc000cc20 comp rx=0 tx=0).stop 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.612+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 shutdown_connections 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.612+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdba806aed0 0x7fdba806d390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.613+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc815da00 0x7fdbc8159860 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.613+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdbc815af30 0x7fdbc8159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.613+0000 7fdbcd9a9640 1 --2- 172.21.15.31:0/2528896723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc806bcd0 0x7fdbc814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.613+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 >> 172.21.15.31:0/2528896723 conn(0x7fdbc8070bf0 msgr2=0x7fdbc80af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.613+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 shutdown_connections 2024-01-19T04:55:53.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.613+0000 7fdbcd9a9640 1 -- 172.21.15.31:0/2528896723 wait complete. 2024-01-19T04:55:53.623 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:55:53.623 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:53.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:53.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.778+0000 7f8bb7382640 1 Processor -- start 2024-01-19T04:55:53.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.778+0000 7f8bb7382640 1 -- start start 2024-01-19T04:55:53.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8bb7382640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 0x7f8bb00d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8bb7382640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb00d06b0 con 0x7f8bb00cfd10 2024-01-19T04:55:53.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8bb7382640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb00d1b90 con 0x7f8bb0160bb0 2024-01-19T04:55:53.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8bb7382640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00d1e70 con 0x7f8bb00d1600 2024-01-19T04:55:53.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8bb491d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8bb00cfd10 0x7f8bb00d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55004/0 (socket says 172.21.15.31:55004) 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8bb491d640 1 -- 172.21.15.31:0/1114059304 learned_addr learned my addr 172.21.15.31:0/1114059304 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8bb591f640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 0x7f8bb00d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.779+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838393401 0 0) 0x7f8bb00d06b0 con 0x7f8bb00cfd10 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b98003520 con 0x7f8bb00cfd10 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 673090485 0 0) 0x7f8b98003520 con 0x7f8bb00cfd10 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 msgr2=0x7f8bb00d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 0x7f8bb00d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 >> v1:172.21.15.137:6789/0 conn(0x7f8bb0160bb0 legacy=0x7f8bb00b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00d2150 con 0x7f8bb00cfd10 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f8ba4001670 con 0x7f8bb00cfd10 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 >> v1:172.21.15.31:6789/0 conn(0x7f8bb00cfd10 legacy=0x7f8bb00d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 0x7f8b980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8b980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b98004d00 0x7f8b98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00d1e70 con 0x7f8b98003f20 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9800ba60 con 0x7f8b98004d00 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9800bc20 con 0x7f8bb00d1600 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb491d640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8b980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb511e640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 0x7f8b980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb591f640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b98004d00 0x7f8b98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb491d640 1 -- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 msgr2=0x7f8b980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb491d640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb00d1600 0x7f8b980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb491d640 1 -- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b98004d00 msgr2=0x7f8b98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb491d640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b98004d00 0x7f8b98005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb491d640 1 -- 172.21.15.31:0/1114059304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98003520 con 0x7f8b98003f20 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.780+0000 7f8bb591f640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b98004d00 0x7f8b98005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.781+0000 7f8bb491d640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8b980047c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c240 tx=0x7f8b980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877d7f25625947dd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.781+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4005ad0 con 0x7f8b98003f20 2024-01-19T04:55:53.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.781+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4005c30 con 0x7f8b98003f20 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.781+0000 7f8baffff640 1 -- 172.21.15.31:0/1114059304 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba402caf0 con 0x7f8b98003f20 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.782+0000 7f8bb7382640 1 -- 172.21.15.31:0/1114059304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 msgr2=0x7f8b980047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.782+0000 7f8bb7382640 1 --2- 172.21.15.31:0/1114059304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8b980047c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c240 tx=0x7f8b980057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.782+0000 7f8bb591f640 1 -- 172.21.15.31:0/1114059304 reap_dead start 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.782+0000 7f8bb7382640 1 -- 172.21.15.31:0/1114059304 shutdown_connections 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.782+0000 7f8bb7382640 1 -- 172.21.15.31:0/1114059304 >> 172.21.15.31:0/1114059304 conn(0x7f8bb009c600 msgr2=0x7f8bb00d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.782+0000 7f8bb7382640 1 -- 172.21.15.31:0/1114059304 shutdown_connections 2024-01-19T04:55:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.782+0000 7f8bb7382640 1 -- 172.21.15.31:0/1114059304 wait complete. 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.783+0000 7f8bb7382640 1 Processor -- start 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.783+0000 7f8bb7382640 1 -- start start 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb7382640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8bb00b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb7382640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bb00cadb0 0x7f8bb00cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb7382640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb0158dd0 0x7f8bb00cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb7382640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00b63a0 con 0x7f8b98003f20 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb7382640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00cb300 con 0x7f8bb00cadb0 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb7382640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00cb5e0 con 0x7f8bb0158dd0 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8bb00b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8bb00b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48370/0 (socket says 172.21.15.31:48370) 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 -- 172.21.15.31:0/811172505 learned_addr learned my addr 172.21.15.31:0/811172505 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb491d640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bb00cadb0 0x7f8bb00cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb591f640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb0158dd0 0x7f8bb00cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 -- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb0158dd0 msgr2=0x7f8bb00cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb0158dd0 0x7f8bb00cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 -- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bb00cadb0 msgr2=0x7f8bb00cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bb00cadb0 0x7f8bb00cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 -- 172.21.15.31:0/811172505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00c8270 con 0x7f8b98003f20 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb491d640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bb00cadb0 0x7f8bb00cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb511e640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8bb00b5e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800f820 tx=0x7f8ba800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8badffb640 1 -- 172.21.15.31:0/811172505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80050d0 con 0x7f8b98003f20 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8badffb640 1 -- 172.21.15.31:0/811172505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8005230 con 0x7f8b98003f20 2024-01-19T04:55:53.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb00c8550 con 0x7f8b98003f20 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.784+0000 7f8badffb640 1 -- 172.21.15.31:0/811172505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800d730 con 0x7f8b98003f20 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.785+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb00cc1f0 con 0x7f8b98003f20 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.786+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb00cb750 con 0x7f8b98003f20 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.786+0000 7f8badffb640 1 -- 172.21.15.31:0/811172505 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8ba800d890 con 0x7f8b98003f20 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.786+0000 7f8badffb640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8b9c06af00 0x7f8b9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.786+0000 7f8badffb640 1 -- 172.21.15.31:0/811172505 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ba8089680 con 0x7f8b98003f20 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.787+0000 7f8bb491d640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8b9c06af00 0x7f8b9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:53.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.787+0000 7f8bb491d640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8b9c06af00 0x7f8b9c06d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4034000 tx=0x7f8ba4009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:53.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.791+0000 7f8badffb640 1 -- 172.21.15.31:0/811172505 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8ba80544b0 con 0x7f8b98003f20 2024-01-19T04:55:53.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.969+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bb015df00 con 0x7f8b98003f20 2024-01-19T04:55:53.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.970+0000 7f8badffb640 1 -- 172.21.15.31:0/811172505 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f8ba8057ad0 con 0x7f8b98003f20 2024-01-19T04:55:53.970 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:53.970 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8b9c06af00 msgr2=0x7f8b9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8b9c06af00 0x7f8b9c06d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4034000 tx=0x7f8ba4009030 comp rx=0 tx=0).stop 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 msgr2=0x7f8bb00b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8bb00b5e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800f820 tx=0x7f8ba800fce0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 shutdown_connections 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8b9c06af00 0x7f8b9c06d3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8bb0158dd0 0x7f8bb00cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bb00cadb0 0x7f8bb00cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 --2- 172.21.15.31:0/811172505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98003f20 0x7f8bb00b5e60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 >> 172.21.15.31:0/811172505 conn(0x7f8bb009c600 msgr2=0x7f8bb015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 shutdown_connections 2024-01-19T04:55:53.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:53.972+0000 7f8bb7382640 1 -- 172.21.15.31:0/811172505 wait complete. 2024-01-19T04:55:53.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:53.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.087+0000 7f613feb4640 1 Processor -- start 2024-01-19T04:55:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.087+0000 7f613feb4640 1 -- start start 2024-01-19T04:55:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.088+0000 7f613feb4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f613813dfc0 0x7f613815f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.088+0000 7f613feb4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f613815edd0 0x7f61380cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.088+0000 7f613feb4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613806c640 con 0x7f61380cf830 2024-01-19T04:55:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.088+0000 7f613feb4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61380cfce0 con 0x7f613813dfc0 2024-01-19T04:55:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.088+0000 7f613feb4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613806c920 con 0x7f613815edd0 2024-01-19T04:55:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.088+0000 7f613d44f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f613815edd0 0x7f61380cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.088+0000 7f613dc50640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f613813dfc0 0x7f613815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613e451640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f61380cf830 0x7f61381408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:41328/0 (socket says 172.21.15.31:41328) 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613dc50640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f613813dfc0 0x7f613815f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:41382/0 (socket says 172.21.15.31:41382) 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613e451640 1 -- 172.21.15.31:0/3523035577 learned_addr learned my addr 172.21.15.31:0/3523035577 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613d44f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f613815edd0 0x7f61380cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:46338/0 (socket says 172.21.15.31:46338) 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613d44f640 1 -- 172.21.15.31:0/3523035577 >> v2:172.21.15.137:3300/0 conn(0x7f613813dfc0 msgr2=0x7f613815f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613d44f640 1 --2- 172.21.15.31:0/3523035577 >> v2:172.21.15.137:3300/0 conn(0x7f613813dfc0 0x7f613815f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613d44f640 1 -- 172.21.15.31:0/3523035577 >> v1:172.21.15.137:6789/0 conn(0x7f61380cf830 legacy=0x7f61381408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.089+0000 7f613d44f640 1 -- 172.21.15.31:0/3523035577 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613806cc00 con 0x7f613815edd0 2024-01-19T04:55:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613d44f640 1 --2- 172.21.15.31:0/3523035577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f613815edd0 0x7f61380cf2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f612c0097a0 tx=0x7f612c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55da2d7b7691633 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613cc4e640 1 -- 172.21.15.31:0/3523035577 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c031830 con 0x7f613815edd0 2024-01-19T04:55:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613cc4e640 1 -- 172.21.15.31:0/3523035577 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c031990 con 0x7f613815edd0 2024-01-19T04:55:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613cc4e640 1 -- 172.21.15.31:0/3523035577 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c031d20 con 0x7f613815edd0 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 -- 172.21.15.31:0/3523035577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f613815edd0 msgr2=0x7f61380cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 --2- 172.21.15.31:0/3523035577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f613815edd0 0x7f61380cf2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f612c0097a0 tx=0x7f612c02b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 -- 172.21.15.31:0/3523035577 shutdown_connections 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 --2- 172.21.15.31:0/3523035577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f613815edd0 0x7f61380cf2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 --2- 172.21.15.31:0/3523035577 >> v2:172.21.15.137:3300/0 conn(0x7f613813dfc0 0x7f613815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 -- 172.21.15.31:0/3523035577 >> 172.21.15.31:0/3523035577 conn(0x7f613806aa00 msgr2=0x7f613806ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 -- 172.21.15.31:0/3523035577 shutdown_connections 2024-01-19T04:55:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.090+0000 7f613feb4640 1 -- 172.21.15.31:0/3523035577 wait complete. 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 Processor -- start 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 -- start start 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61380cf830 0x7f6138137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613813dfc0 0x7f61380ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f613815edd0 0x7f61380afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380af070 con 0x7f613815edd0 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380af350 con 0x7f613813dfc0 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613feb4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61380af630 con 0x7f61380cf830 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613e451640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f613815edd0 0x7f61380afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613e451640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f613815edd0 0x7f61380afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48380/0 (socket says 172.21.15.31:48380) 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.092+0000 7f613e451640 1 -- 172.21.15.31:0/518034279 learned_addr learned my addr 172.21.15.31:0/518034279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613d44f640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613813dfc0 0x7f61380ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613dc50640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61380cf830 0x7f6138137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613e451640 1 -- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61380cf830 msgr2=0x7f6138137a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613e451640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61380cf830 0x7f6138137a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613e451640 1 -- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613813dfc0 msgr2=0x7f61380ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613e451640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613813dfc0 0x7f61380ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613e451640 1 -- 172.21.15.31:0/518034279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c009400 con 0x7f613815edd0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613e451640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f613815edd0 0x7f61380afe10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f613000c3c0 tx=0x7f613000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f61367fc640 1 -- 172.21.15.31:0/518034279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130005ce0 con 0x7f613815edd0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f61367fc640 1 -- 172.21.15.31:0/518034279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6130005e40 con 0x7f613815edd0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f61367fc640 1 -- 172.21.15.31:0/518034279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61300072f0 con 0x7f613815edd0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61380af910 con 0x7f613815edd0 2024-01-19T04:55:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.093+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61380aecd0 con 0x7f613815edd0 2024-01-19T04:55:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.094+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61380acba0 con 0x7f613815edd0 2024-01-19T04:55:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.095+0000 7f61367fc640 1 -- 172.21.15.31:0/518034279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6130007510 con 0x7f613815edd0 2024-01-19T04:55:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.095+0000 7f61367fc640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f612406aed0 0x7f612406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.095+0000 7f61367fc640 1 -- 172.21.15.31:0/518034279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6130088a70 con 0x7f613815edd0 2024-01-19T04:55:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.095+0000 7f613dc50640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f612406aed0 0x7f612406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.096+0000 7f613dc50640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f612406aed0 0x7f612406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f61280113e0 tx=0x7f612800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.099+0000 7f61367fc640 1 -- 172.21.15.31:0/518034279 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f61380acba0 con 0x7f613815edd0 2024-01-19T04:55:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.280+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61380acba0 con 0x7f613815edd0 2024-01-19T04:55:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.280+0000 7f61367fc640 1 -- 172.21.15.31:0/518034279 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61380acba0 con 0x7f613815edd0 2024-01-19T04:55:54.280 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f612406aed0 msgr2=0x7f612406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f612406aed0 0x7f612406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f61280113e0 tx=0x7f612800b040 comp rx=0 tx=0).stop 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f613815edd0 msgr2=0x7f61380afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f613815edd0 0x7f61380afe10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f613000c3c0 tx=0x7f613000c880 comp rx=0 tx=0).stop 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 shutdown_connections 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f612406aed0 0x7f612406d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f613815edd0 0x7f61380afe10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613813dfc0 0x7f61380ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 --2- 172.21.15.31:0/518034279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61380cf830 0x7f6138137a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 >> 172.21.15.31:0/518034279 conn(0x7f613806aa00 msgr2=0x7f6138151b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 shutdown_connections 2024-01-19T04:55:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.282+0000 7f613feb4640 1 -- 172.21.15.31:0/518034279 wait complete. 2024-01-19T04:55:54.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.397+0000 7eff73133640 1 Processor -- start 2024-01-19T04:55:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.397+0000 7eff73133640 1 -- start start 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff73133640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff6c15d3a0 0x7eff6c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff73133640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7eff6c0cf570 0x7eff6c161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff73133640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d780 0x7eff6c161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff73133640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c0cf9f0 con 0x7eff6c15d3a0 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff73133640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c0cfcd0 con 0x7eff6c0cf570 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff73133640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0cffb0 con 0x7eff6c15d780 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff6c15d3a0 0x7eff6c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff6c15d3a0 0x7eff6c06c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48388/0 (socket says 172.21.15.31:48388) 2024-01-19T04:55:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 -- 172.21.15.31:0/1555780969 learned_addr learned my addr 172.21.15.31:0/1555780969 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 -- 172.21.15.31:0/1555780969 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d780 msgr2=0x7eff6c161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff716d0640 1 --2- 172.21.15.31:0/1555780969 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d780 0x7eff6c161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 --2- 172.21.15.31:0/1555780969 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d780 0x7eff6c161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 -- 172.21.15.31:0/1555780969 >> v2:172.21.15.137:3300/0 conn(0x7eff6c0cf570 msgr2=0x7eff6c161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 --2- 172.21.15.31:0/1555780969 >> v2:172.21.15.137:3300/0 conn(0x7eff6c0cf570 0x7eff6c161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.398+0000 7eff70ecf640 1 -- 172.21.15.31:0/1555780969 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0d0290 con 0x7eff6c15d3a0 2024-01-19T04:55:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.399+0000 7eff70ecf640 1 --2- 172.21.15.31:0/1555780969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d3a0 0x7eff6c06c4e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0b71b0 tx=0x7eff6402bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304cc92ec125e96e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.399+0000 7eff6b7fe640 1 -- 172.21.15.31:0/1555780969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6402cac0 con 0x7eff6c15d3a0 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.399+0000 7eff6b7fe640 1 -- 172.21.15.31:0/1555780969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6402cc20 con 0x7eff6c15d3a0 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.399+0000 7eff6b7fe640 1 -- 172.21.15.31:0/1555780969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff640343f0 con 0x7eff6c15d3a0 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 -- 172.21.15.31:0/1555780969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d3a0 msgr2=0x7eff6c06c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 --2- 172.21.15.31:0/1555780969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d3a0 0x7eff6c06c4e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0b71b0 tx=0x7eff6402bf20 comp rx=0 tx=0).stop 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 -- 172.21.15.31:0/1555780969 shutdown_connections 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 --2- 172.21.15.31:0/1555780969 >> v2:172.21.15.137:3300/0 conn(0x7eff6c0cf570 0x7eff6c161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 --2- 172.21.15.31:0/1555780969 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d780 0x7eff6c161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 --2- 172.21.15.31:0/1555780969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d3a0 0x7eff6c06c4e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 -- 172.21.15.31:0/1555780969 >> 172.21.15.31:0/1555780969 conn(0x7eff6c0d1100 msgr2=0x7eff6c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 -- 172.21.15.31:0/1555780969 shutdown_connections 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.400+0000 7eff73133640 1 -- 172.21.15.31:0/1555780969 wait complete. 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.401+0000 7eff73133640 1 Processor -- start 2024-01-19T04:55:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.401+0000 7eff73133640 1 -- start start 2024-01-19T04:55:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff73133640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff6c0cf570 0x7eff6c0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff73133640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d3a0 0x7eff6c0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff73133640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d780 0x7eff6c1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff73133640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0b6bc0 con 0x7eff6c15d780 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff73133640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0b4690 con 0x7eff6c0cf570 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff73133640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0b5210 con 0x7eff6c15d3a0 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d780 0x7eff6c1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d780 0x7eff6c1546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48404/0 (socket says 172.21.15.31:48404) 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 -- 172.21.15.31:0/3940493687 learned_addr learned my addr 172.21.15.31:0/3940493687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff6bfff640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d3a0 0x7eff6c0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff70ecf640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff6c0cf570 0x7eff6c0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 -- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d3a0 msgr2=0x7eff6c0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d3a0 0x7eff6c0b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 -- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff6c0cf570 msgr2=0x7eff6c0b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff6c0cf570 0x7eff6c0b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff716d0640 1 -- 172.21.15.31:0/3940493687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff64009b30 con 0x7eff6c15d780 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff70ecf640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff6c0cf570 0x7eff6c0b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.402+0000 7eff6bfff640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d3a0 0x7eff6c0b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.403+0000 7eff716d0640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d780 0x7eff6c1546d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c580 tx=0x7eff5c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.403+0000 7eff697fa640 1 -- 172.21.15.31:0/3940493687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004220 con 0x7eff6c15d780 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.403+0000 7eff697fa640 1 -- 172.21.15.31:0/3940493687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c004380 con 0x7eff6c15d780 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.403+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c154c10 con 0x7eff6c15d780 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.403+0000 7eff697fa640 1 -- 172.21.15.31:0/3940493687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c003830 con 0x7eff6c15d780 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.403+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c0b4bc0 con 0x7eff6c15d780 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.404+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c074900 con 0x7eff6c15d780 2024-01-19T04:55:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.404+0000 7eff697fa640 1 -- 172.21.15.31:0/3940493687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7eff5c003990 con 0x7eff6c15d780 2024-01-19T04:55:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.404+0000 7eff697fa640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7eff5806aed0 0x7eff5806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.405+0000 7eff697fa640 1 -- 172.21.15.31:0/3940493687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff5c089300 con 0x7eff6c15d780 2024-01-19T04:55:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.405+0000 7eff70ecf640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7eff5806aed0 0x7eff5806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.405+0000 7eff70ecf640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7eff5806aed0 0x7eff5806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff64035000 tx=0x7eff64030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.409+0000 7eff697fa640 1 -- 172.21.15.31:0/3940493687 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7eff5c0541e0 con 0x7eff6c15d780 2024-01-19T04:55:54.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.587+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff6c0b5350 con 0x7eff6c15d780 2024-01-19T04:55:54.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.588+0000 7eff697fa640 1 -- 172.21.15.31:0/3940493687 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff5c057800 con 0x7eff6c15d780 2024-01-19T04:55:54.588 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7eff5806aed0 msgr2=0x7eff5806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7eff5806aed0 0x7eff5806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff64035000 tx=0x7eff64030000 comp rx=0 tx=0).stop 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d780 msgr2=0x7eff6c1546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d780 0x7eff6c1546d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c580 tx=0x7eff5c00ca40 comp rx=0 tx=0).stop 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 shutdown_connections 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7eff5806aed0 0x7eff5806d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff6c15d780 0x7eff6c1546d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7eff6c15d3a0 0x7eff6c0b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 --2- 172.21.15.31:0/3940493687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff6c0cf570 0x7eff6c0b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 >> 172.21.15.31:0/3940493687 conn(0x7eff6c0d1100 msgr2=0x7eff6c15b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 shutdown_connections 2024-01-19T04:55:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.590+0000 7eff73133640 1 -- 172.21.15.31:0/3940493687 wait complete. 2024-01-19T04:55:54.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:54.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.704+0000 7fcf56ac2640 1 Processor -- start 2024-01-19T04:55:54.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.704+0000 7fcf56ac2640 1 -- start start 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf56ac2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf5015d570 0x7fcf5006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf56ac2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500d1820 0x7fcf5006c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf56ac2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcf500cbed0 0x7fcf500cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf56ac2640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf500b0450 con 0x7fcf5015d570 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf56ac2640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf500cb760 con 0x7fcf500cbed0 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf56ac2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf500cba40 con 0x7fcf500d1820 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf5485e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf5015d570 0x7fcf5006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf5485e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf5015d570 0x7fcf5006bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48416/0 (socket says 172.21.15.31:48416) 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.705+0000 7fcf5485e640 1 -- 172.21.15.31:0/2268590980 learned_addr learned my addr 172.21.15.31:0/2268590980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf4ffff640 1 --2- 172.21.15.31:0/2268590980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500d1820 0x7fcf5006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf5485e640 1 -- 172.21.15.31:0/2268590980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500d1820 msgr2=0x7fcf5006c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf5505f640 1 --2- 172.21.15.31:0/2268590980 >> v2:172.21.15.137:3300/0 conn(0x7fcf500cbed0 0x7fcf500cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf5485e640 1 --2- 172.21.15.31:0/2268590980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500d1820 0x7fcf5006c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf5485e640 1 -- 172.21.15.31:0/2268590980 >> v2:172.21.15.137:3300/0 conn(0x7fcf500cbed0 msgr2=0x7fcf500cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf5485e640 1 --2- 172.21.15.31:0/2268590980 >> v2:172.21.15.137:3300/0 conn(0x7fcf500cbed0 0x7fcf500cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf5485e640 1 -- 172.21.15.31:0/2268590980 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf500ce630 con 0x7fcf5015d570 2024-01-19T04:55:54.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.706+0000 7fcf5485e640 1 --2- 172.21.15.31:0/2268590980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5006bfd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcf50072450 tx=0x7fcf4802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dcd1a0b31c7e8d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf4f7fe640 1 -- 172.21.15.31:0/2268590980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4802c9e0 con 0x7fcf5015d570 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf4f7fe640 1 -- 172.21.15.31:0/2268590980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4802cb40 con 0x7fcf5015d570 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf4f7fe640 1 -- 172.21.15.31:0/2268590980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf480343f0 con 0x7fcf5015d570 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2268590980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 msgr2=0x7fcf5006bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2268590980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5006bfd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcf50072450 tx=0x7fcf4802be40 comp rx=0 tx=0).stop 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2268590980 shutdown_connections 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2268590980 >> v2:172.21.15.137:3300/0 conn(0x7fcf500cbed0 0x7fcf500cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2268590980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500d1820 0x7fcf5006c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2268590980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5006bfd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2268590980 >> 172.21.15.31:0/2268590980 conn(0x7fcf5008afe0 msgr2=0x7fcf500d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2268590980 shutdown_connections 2024-01-19T04:55:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.707+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2268590980 wait complete. 2024-01-19T04:55:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.708+0000 7fcf56ac2640 1 Processor -- start 2024-01-19T04:55:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.708+0000 7fcf56ac2640 1 -- start start 2024-01-19T04:55:54.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf56ac2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500cbed0 0x7fcf50159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf56ac2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf500d1820 0x7fcf500c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf56ac2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf56ac2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf500c9f70 con 0x7fcf5015d570 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf56ac2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf50159790 con 0x7fcf500d1820 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf56ac2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf500c88a0 con 0x7fcf500cbed0 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf5505f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf5505f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5015afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48426/0 (socket says 172.21.15.31:48426) 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.709+0000 7fcf5505f640 1 -- 172.21.15.31:0/2276306124 learned_addr learned my addr 172.21.15.31:0/2276306124 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5485e640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500cbed0 0x7fcf50159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf4ffff640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf500d1820 0x7fcf500c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5505f640 1 -- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500cbed0 msgr2=0x7fcf50159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5505f640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500cbed0 0x7fcf50159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5505f640 1 -- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf500d1820 msgr2=0x7fcf500c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5505f640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf500d1820 0x7fcf500c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5505f640 1 -- 172.21.15.31:0/2276306124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf48009b30 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5485e640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500cbed0 0x7fcf50159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf4ffff640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf500d1820 0x7fcf500c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf5505f640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5015afc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c4c0 tx=0x7fcf4000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf4d7fa640 1 -- 172.21.15.31:0/2276306124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40004220 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf4d7fa640 1 -- 172.21.15.31:0/2276306124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf40004380 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf500c5cf0 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf4d7fa640 1 -- 172.21.15.31:0/2276306124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40003810 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.710+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf500c5f90 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.711+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5006b920 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.712+0000 7fcf4d7fa640 1 -- 172.21.15.31:0/2276306124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fcf40003970 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.712+0000 7fcf4d7fa640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf3c06aed0 0x7fcf3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.712+0000 7fcf4d7fa640 1 -- 172.21.15.31:0/2276306124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf400891e0 con 0x7fcf5015d570 2024-01-19T04:55:54.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.712+0000 7fcf5485e640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf3c06aed0 0x7fcf3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:54.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.713+0000 7fcf5485e640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf3c06aed0 0x7fcf3c06d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf48035000 tx=0x7fcf48030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:54.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.716+0000 7fcf4d7fa640 1 -- 172.21.15.31:0/2276306124 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fcf40054140 con 0x7fcf5015d570 2024-01-19T04:55:54.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.894+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf50074900 con 0x7fcf5015d570 2024-01-19T04:55:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.894+0000 7fcf4d7fa640 1 -- 172.21.15.31:0/2276306124 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf40057760 con 0x7fcf5015d570 2024-01-19T04:55:54.894 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf3c06aed0 msgr2=0x7fcf3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf3c06aed0 0x7fcf3c06d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf48035000 tx=0x7fcf48030000 comp rx=0 tx=0).stop 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 msgr2=0x7fcf5015afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5015afc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c4c0 tx=0x7fcf4000c980 comp rx=0 tx=0).stop 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 shutdown_connections 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcf3c06aed0 0x7fcf3c06d390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5015d570 0x7fcf5015afc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf500d1820 0x7fcf500c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 --2- 172.21.15.31:0/2276306124 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcf500cbed0 0x7fcf50159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 >> 172.21.15.31:0/2276306124 conn(0x7fcf5008afe0 msgr2=0x7fcf500add90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 shutdown_connections 2024-01-19T04:55:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:54.896+0000 7fcf56ac2640 1 -- 172.21.15.31:0/2276306124 wait complete. 2024-01-19T04:55:54.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:55:55.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.008+0000 7fd3378ce640 1 Processor -- start 2024-01-19T04:55:55.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.009+0000 7fd3378ce640 1 -- start start 2024-01-19T04:55:55.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.009+0000 7fd3378ce640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd33015b6b0 0x7fd3300d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.009+0000 7fd3378ce640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd3300cd2e0 0x7fd3300cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.009+0000 7fd3378ce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd3300ceee0 0x7fd3300cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.009+0000 7fd3378ce640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300d03b0 con 0x7fd33015b6b0 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.009+0000 7fd3378ce640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300cce50 con 0x7fd3300cd2e0 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.009+0000 7fd3378ce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300cc240 con 0x7fd3300ceee0 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd33015b6b0 0x7fd3300d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd33015b6b0 0x7fd3300d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48436/0 (socket says 172.21.15.31:48436) 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 -- 172.21.15.31:0/688420752 learned_addr learned my addr 172.21.15.31:0/688420752 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd334e69640 1 --2- 172.21.15.31:0/688420752 >> v2:172.21.15.137:3300/0 conn(0x7fd3300cd2e0 0x7fd3300cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd335e6b640 1 --2- 172.21.15.31:0/688420752 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd3300ceee0 0x7fd3300cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 -- 172.21.15.31:0/688420752 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd3300ceee0 msgr2=0x7fd3300cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 --2- 172.21.15.31:0/688420752 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd3300ceee0 0x7fd3300cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 -- 172.21.15.31:0/688420752 >> v2:172.21.15.137:3300/0 conn(0x7fd3300cd2e0 msgr2=0x7fd3300cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 --2- 172.21.15.31:0/688420752 >> v2:172.21.15.137:3300/0 conn(0x7fd3300cd2e0 0x7fd3300cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd33566a640 1 -- 172.21.15.31:0/688420752 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300cc520 con 0x7fd33015b6b0 2024-01-19T04:55:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.010+0000 7fd334e69640 1 --2- 172.21.15.31:0/688420752 >> v2:172.21.15.137:3300/0 conn(0x7fd3300cd2e0 0x7fd3300cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd33566a640 1 --2- 172.21.15.31:0/688420752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd33015b6b0 0x7fd3300d1c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd3300729e0 tx=0x7fd32802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870f7eaaf84e7483 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd32ffff640 1 -- 172.21.15.31:0/688420752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328004230 con 0x7fd33015b6b0 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd32ffff640 1 -- 172.21.15.31:0/688420752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd328004390 con 0x7fd33015b6b0 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd32ffff640 1 -- 172.21.15.31:0/688420752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328030840 con 0x7fd33015b6b0 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 -- 172.21.15.31:0/688420752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd33015b6b0 msgr2=0x7fd3300d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 --2- 172.21.15.31:0/688420752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd33015b6b0 0x7fd3300d1c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd3300729e0 tx=0x7fd32802b980 comp rx=0 tx=0).stop 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 -- 172.21.15.31:0/688420752 shutdown_connections 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 --2- 172.21.15.31:0/688420752 >> v2:172.21.15.137:3300/0 conn(0x7fd3300cd2e0 0x7fd3300cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 --2- 172.21.15.31:0/688420752 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd3300ceee0 0x7fd3300cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 --2- 172.21.15.31:0/688420752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd33015b6b0 0x7fd3300d1c20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 -- 172.21.15.31:0/688420752 >> 172.21.15.31:0/688420752 conn(0x7fd3300d1100 msgr2=0x7fd3300d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 -- 172.21.15.31:0/688420752 shutdown_connections 2024-01-19T04:55:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.011+0000 7fd3378ce640 1 -- 172.21.15.31:0/688420752 wait complete. 2024-01-19T04:55:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.012+0000 7fd3378ce640 1 Processor -- start 2024-01-19T04:55:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd3378ce640 1 -- start start 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd3378ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3300cd2e0 0x7fd330158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd3378ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3300ceee0 0x7fd33015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd3378ce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd33015b6b0 0x7fd3300c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd3378ce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c5640 con 0x7fd3300ceee0 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd3378ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c5890 con 0x7fd3300cd2e0 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd3378ce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c5b70 con 0x7fd33015b6b0 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd334e69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3300ceee0 0x7fd33015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd334e69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3300ceee0 0x7fd33015d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48446/0 (socket says 172.21.15.31:48446) 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd334e69640 1 -- 172.21.15.31:0/1308883611 learned_addr learned my addr 172.21.15.31:0/1308883611 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.013+0000 7fd335e6b640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd33015b6b0 0x7fd3300c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd33566a640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3300cd2e0 0x7fd330158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd334e69640 1 -- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd33015b6b0 msgr2=0x7fd3300c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd334e69640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd33015b6b0 0x7fd3300c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd334e69640 1 -- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3300cd2e0 msgr2=0x7fd330158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd334e69640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3300cd2e0 0x7fd330158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd334e69640 1 -- 172.21.15.31:0/1308883611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009460 con 0x7fd3300ceee0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd33566a640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3300cd2e0 0x7fd330158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd335e6b640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd33015b6b0 0x7fd3300c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd334e69640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3300ceee0 0x7fd33015d7f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd32400e7e0 tx=0x7fd32400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd32dffb640 1 -- 172.21.15.31:0/1308883611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324004030 con 0x7fd3300ceee0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd32dffb640 1 -- 172.21.15.31:0/1308883611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324004190 con 0x7fd3300ceee0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3300c5e50 con 0x7fd3300ceee0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd32dffb640 1 -- 172.21.15.31:0/1308883611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324004520 con 0x7fd3300ceee0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.014+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33015dd30 con 0x7fd3300ceee0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.015+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3300d01d0 con 0x7fd3300ceee0 2024-01-19T04:55:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.015+0000 7fd32dffb640 1 -- 172.21.15.31:0/1308883611 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd32400c6d0 con 0x7fd3300ceee0 2024-01-19T04:55:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.016+0000 7fd32dffb640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd31c06af00 0x7fd31c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.016+0000 7fd32dffb640 1 -- 172.21.15.31:0/1308883611 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd324089360 con 0x7fd3300ceee0 2024-01-19T04:55:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.016+0000 7fd33566a640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd31c06af00 0x7fd31c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.017+0000 7fd33566a640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd31c06af00 0x7fd31c06d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd328035000 tx=0x7fd328008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.020+0000 7fd32dffb640 1 -- 172.21.15.31:0/1308883611 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd3240541a0 con 0x7fd3300ceee0 2024-01-19T04:55:55.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.210+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd330074900 con 0x7fd3300ceee0 2024-01-19T04:55:55.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.210+0000 7fd32dffb640 1 -- 172.21.15.31:0/1308883611 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd3240577c0 con 0x7fd3300ceee0 2024-01-19T04:55:55.210 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:55:55.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd31c06af00 msgr2=0x7fd31c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd31c06af00 0x7fd31c06d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd328035000 tx=0x7fd328008030 comp rx=0 tx=0).stop 2024-01-19T04:55:55.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3300ceee0 msgr2=0x7fd33015d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3300ceee0 0x7fd33015d7f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd32400e7e0 tx=0x7fd32400eca0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 shutdown_connections 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd31c06af00 0x7fd31c06d3c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd33015b6b0 0x7fd3300c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3300ceee0 0x7fd33015d7f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 --2- 172.21.15.31:0/1308883611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3300cd2e0 0x7fd330158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 >> 172.21.15.31:0/1308883611 conn(0x7fd3300d1100 msgr2=0x7fd33006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 shutdown_connections 2024-01-19T04:55:55.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.213+0000 7fd3378ce640 1 -- 172.21.15.31:0/1308883611 wait complete. 2024-01-19T04:55:55.223 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:55:55.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:55.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-19T04:55:55.249 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:55:55.250 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:55:55.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:55.250 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:55:55.315 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:55:55.316 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:55.334 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:55:55.334 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:55.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:55.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.494+0000 7f28e7161640 1 Processor -- start 2024-01-19T04:55:55.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.494+0000 7f28e7161640 1 -- start start 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e7161640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f28e0160bb0 0x7f28e00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e7161640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 0x7f28e00d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e7161640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e00d06b0 con 0x7f28e00cfd10 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e7161640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e00b66b0 con 0x7f28e0160bb0 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e7161640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e00d1b90 con 0x7f28e00d1600 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28dffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f28e00cfd10 0x7f28e00d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55016/0 (socket says 172.21.15.31:55016) 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28dffff640 1 -- 172.21.15.31:0/1661530223 learned_addr learned my addr 172.21.15.31:0/1661530223 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e4efd640 1 --2- 172.21.15.31:0/1661530223 >> v2:172.21.15.137:3300/0 conn(0x7f28e0160bb0 0x7f28e00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e56fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 0x7f28e00d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617891148 0 0) 0x7f28e00d06b0 con 0x7f28e00cfd10 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d0003520 con 0x7f28e00cfd10 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148582313 0 0) 0x7f28d0003520 con 0x7f28e00cfd10 2024-01-19T04:55:55.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 msgr2=0x7f28e00d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 0x7f28e00d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 >> v2:172.21.15.137:3300/0 conn(0x7f28e0160bb0 msgr2=0x7f28e00b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 --2- 172.21.15.31:0/1661530223 >> v2:172.21.15.137:3300/0 conn(0x7f28e0160bb0 0x7f28e00b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e00d1e70 con 0x7f28e00cfd10 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f28cc001670 con 0x7f28e00cfd10 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 >> v1:172.21.15.31:6789/0 conn(0x7f28e00cfd10 legacy=0x7f28e00d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 0x7f28d0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d0003ef0 0x7f28d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0004d00 0x7f28d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e00b66b0 con 0x7f28d0004d00 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e00d1b90 con 0x7f28d0003ef0 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d000b980 con 0x7f28e00d1600 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e56fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0004d00 0x7f28d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.495+0000 7f28e4efd640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 0x7f28d0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.496+0000 7f28dffff640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d0003ef0 0x7f28d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.496+0000 7f28e56fe640 1 -- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 msgr2=0x7f28d0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.496+0000 7f28e56fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00d1600 0x7f28d0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.496+0000 7f28e56fe640 1 -- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d0003ef0 msgr2=0x7f28d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.496+0000 7f28e56fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d0003ef0 0x7f28d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.496+0000 7f28e56fe640 1 -- 172.21.15.31:0/1661530223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0003520 con 0x7f28d0004d00 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.496+0000 7f28e56fe640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0004d00 0x7f28d0005180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009fc0 tx=0x7f28d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2127fa690ab2c2a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d802cc00 con 0x7f28d0004d00 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d800f040 con 0x7f28d0004d00 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28df7fe640 1 -- 172.21.15.31:0/1661530223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8034440 con 0x7f28d0004d00 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28e7161640 1 -- 172.21.15.31:0/1661530223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0004d00 msgr2=0x7f28d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28e7161640 1 --2- 172.21.15.31:0/1661530223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0004d00 0x7f28d0005180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009fc0 tx=0x7f28d00057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28e56fe640 1 -- 172.21.15.31:0/1661530223 reap_dead start 2024-01-19T04:55:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28e7161640 1 -- 172.21.15.31:0/1661530223 shutdown_connections 2024-01-19T04:55:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28e7161640 1 -- 172.21.15.31:0/1661530223 >> 172.21.15.31:0/1661530223 conn(0x7f28e009c600 msgr2=0x7f28e00d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28e7161640 1 -- 172.21.15.31:0/1661530223 shutdown_connections 2024-01-19T04:55:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.497+0000 7f28e7161640 1 -- 172.21.15.31:0/1661530223 wait complete. 2024-01-19T04:55:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.498+0000 7f28e7161640 1 Processor -- start 2024-01-19T04:55:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.498+0000 7f28e7161640 1 -- start start 2024-01-19T04:55:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e7161640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00cc830 0x7f28e01f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e7161640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e015cf50 0x7f28e015a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e7161640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e00ccc10 0x7f28e015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e7161640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01f7540 con 0x7f28e00ccc10 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e7161640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e015d4a0 con 0x7f28e015cf50 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e7161640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e015d780 con 0x7f28e00cc830 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e56fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e00ccc10 0x7f28e015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e56fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e00ccc10 0x7f28e015a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48470/0 (socket says 172.21.15.31:48470) 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e56fe640 1 -- 172.21.15.31:0/2475233849 learned_addr learned my addr 172.21.15.31:0/2475233849 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.499+0000 7f28e4efd640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00cc830 0x7f28e01f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28dffff640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e015cf50 0x7f28e015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e56fe640 1 -- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00cc830 msgr2=0x7f28e01f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e56fe640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00cc830 0x7f28e01f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e56fe640 1 -- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e015cf50 msgr2=0x7f28e015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e56fe640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e015cf50 0x7f28e015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e56fe640 1 -- 172.21.15.31:0/2475233849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e015da60 con 0x7f28e00ccc10 2024-01-19T04:55:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e56fe640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e00ccc10 0x7f28e015a9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f28d8037040 tx=0x7f28d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28dd7fa640 1 -- 172.21.15.31:0/2475233849 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8043070 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28dd7fa640 1 -- 172.21.15.31:0/2475233849 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d802de90 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28dd7fa640 1 -- 172.21.15.31:0/2475233849 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8034d90 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e015dd40 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.500+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e00c7030 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.501+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e015df30 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.502+0000 7f28dd7fa640 1 -- 172.21.15.31:0/2475233849 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f28d803e440 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.502+0000 7f28dd7fa640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f28c406aed0 0x7f28c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.502+0000 7f28dd7fa640 1 -- 172.21.15.31:0/2475233849 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28d80ab0d0 con 0x7f28e00ccc10 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.502+0000 7f28e4efd640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f28c406aed0 0x7f28c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.503+0000 7f28e4efd640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f28c406aed0 0x7f28c406d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009c40 tx=0x7f28d4009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.506+0000 7f28dd7fa640 1 -- 172.21.15.31:0/2475233849 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f28d80760f0 con 0x7f28e00ccc10 2024-01-19T04:55:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.685+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28e00696e0 con 0x7f28e00ccc10 2024-01-19T04:55:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.686+0000 7f28dd7fa640 1 -- 172.21.15.31:0/2475233849 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f28d8079710 con 0x7f28e00ccc10 2024-01-19T04:55:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:55.686 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f28c406aed0 msgr2=0x7f28c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f28c406aed0 0x7f28c406d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009c40 tx=0x7f28d4009190 comp rx=0 tx=0).stop 2024-01-19T04:55:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e00ccc10 msgr2=0x7f28e015a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e00ccc10 0x7f28e015a9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f28d8037040 tx=0x7f28d8009750 comp rx=0 tx=0).stop 2024-01-19T04:55:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 shutdown_connections 2024-01-19T04:55:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f28c406aed0 0x7f28c406d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28e00ccc10 0x7f28e015a9d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e015cf50 0x7f28e015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 --2- 172.21.15.31:0/2475233849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28e00cc830 0x7f28e01f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 >> 172.21.15.31:0/2475233849 conn(0x7f28e009c600 msgr2=0x7f28e006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 shutdown_connections 2024-01-19T04:55:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.688+0000 7f28e7161640 1 -- 172.21.15.31:0/2475233849 wait complete. 2024-01-19T04:55:55.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:55.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.804+0000 7fe9e6bb0640 1 Processor -- start 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.804+0000 7fe9e6bb0640 1 -- start start 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.804+0000 7fe9e6bb0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9e00cf570 0x7fe9e0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.804+0000 7fe9e6bb0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e015d780 0x7fe9e0161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.804+0000 7fe9e6bb0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e00cf9f0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.804+0000 7fe9e6bb0640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e00cfcd0 con 0x7fe9e00cf570 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.804+0000 7fe9e6bb0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00cffb0 con 0x7fe9e015d780 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9dffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9e00cf570 0x7fe9e0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9dffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9e00cf570 0x7fe9e0161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48474/0 (socket says 172.21.15.31:48474) 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9dffff640 1 -- 172.21.15.31:0/950247826 learned_addr learned my addr 172.21.15.31:0/950247826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9e514d640 1 --2- 172.21.15.31:0/950247826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e015d780 0x7fe9e0161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9dffff640 1 -- 172.21.15.31:0/950247826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e015d780 msgr2=0x7fe9e0161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9dffff640 1 --2- 172.21.15.31:0/950247826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e015d780 0x7fe9e0161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9dffff640 1 -- 172.21.15.31:0/950247826 >> v1:172.21.15.137:6789/0 conn(0x7fe9e015d3a0 legacy=0x7fe9e006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9dffff640 1 -- 172.21.15.31:0/950247826 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00d0290 con 0x7fe9e00cf570 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.805+0000 7fe9e514d640 1 --2- 172.21.15.31:0/950247826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e015d780 0x7fe9e0161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9dffff640 1 --2- 172.21.15.31:0/950247826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00cf570 0x7fe9e0161150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009870 tx=0x7fe9cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5177bf46537fb80b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9df7fe640 1 -- 172.21.15.31:0/950247826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc031640 con 0x7fe9e00cf570 2024-01-19T04:55:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9df7fe640 1 -- 172.21.15.31:0/950247826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc0317a0 con 0x7fe9e00cf570 2024-01-19T04:55:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9df7fe640 1 -- 172.21.15.31:0/950247826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc031b30 con 0x7fe9e00cf570 2024-01-19T04:55:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/950247826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00cf570 msgr2=0x7fe9e0161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/950247826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00cf570 0x7fe9e0161150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009870 tx=0x7fe9cc02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/950247826 shutdown_connections 2024-01-19T04:55:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/950247826 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e015d780 0x7fe9e0161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/950247826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00cf570 0x7fe9e0161150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/950247826 >> 172.21.15.31:0/950247826 conn(0x7fe9e00d1100 msgr2=0x7fe9e00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/950247826 shutdown_connections 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.806+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/950247826 wait complete. 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 Processor -- start 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 -- start start 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e00cf570 0x7fe9e01e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e015d3a0 0x7fe9e01535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9e015d780 0x7fe9e0155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01e7310 con 0x7fe9e015d3a0 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01606a0 con 0x7fe9e015d780 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.808+0000 7fe9e6bb0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e015fe90 con 0x7fe9e00cf570 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e015d3a0 0x7fe9e01535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e015d3a0 0x7fe9e01535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48482/0 (socket says 172.21.15.31:48482) 2024-01-19T04:55:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 -- 172.21.15.31:0/2472581251 learned_addr learned my addr 172.21.15.31:0/2472581251 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9e494c640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e00cf570 0x7fe9e01e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9e514d640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9e015d780 0x7fe9e0155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 -- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e00cf570 msgr2=0x7fe9e01e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e00cf570 0x7fe9e01e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 -- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9e015d780 msgr2=0x7fe9e0155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9e015d780 0x7fe9e0155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 -- 172.21.15.31:0/2472581251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0094d0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9e514d640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9e015d780 0x7fe9e0155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9e494c640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e00cf570 0x7fe9e01e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dffff640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e015d3a0 0x7fe9e01535c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc000c00 tx=0x7fe9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dd7fa640 1 -- 172.21.15.31:0/2472581251 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc03ac60 con 0x7fe9e015d3a0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dd7fa640 1 -- 172.21.15.31:0/2472581251 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc03ade0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e0154fb0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9dd7fa640 1 -- 172.21.15.31:0/2472581251 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc044630 con 0x7fe9e015d3a0 2024-01-19T04:55:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.809+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01603c0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.810+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e006bde0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.811+0000 7fe9dd7fa640 1 -- 172.21.15.31:0/2472581251 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe9cc02f050 con 0x7fe9e015d3a0 2024-01-19T04:55:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.811+0000 7fe9dd7fa640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe9c406aed0 0x7fe9c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.811+0000 7fe9dd7fa640 1 -- 172.21.15.31:0/2472581251 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9cc0ab7b0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.811+0000 7fe9e494c640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe9c406aed0 0x7fe9c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.812+0000 7fe9e494c640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe9c406aed0 0x7fe9c406d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4003ee0 tx=0x7fe9d4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.815+0000 7fe9dd7fa640 1 -- 172.21.15.31:0/2472581251 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe9cc076680 con 0x7fe9e015d3a0 2024-01-19T04:55:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.991+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9e00b0330 con 0x7fe9e015d3a0 2024-01-19T04:55:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.992+0000 7fe9dd7fa640 1 -- 172.21.15.31:0/2472581251 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9cc079ca0 con 0x7fe9e015d3a0 2024-01-19T04:55:55.991 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.993+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe9c406aed0 msgr2=0x7fe9c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.993+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe9c406aed0 0x7fe9c406d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4003ee0 tx=0x7fe9d4009040 comp rx=0 tx=0).stop 2024-01-19T04:55:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.993+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e015d3a0 msgr2=0x7fe9e01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.993+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e015d3a0 0x7fe9e01535c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc000c00 tx=0x7fe9cc0046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 shutdown_connections 2024-01-19T04:55:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe9c406aed0 0x7fe9c406d390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9e015d780 0x7fe9e0155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e015d3a0 0x7fe9e01535c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 --2- 172.21.15.31:0/2472581251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9e00cf570 0x7fe9e01e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 >> 172.21.15.31:0/2472581251 conn(0x7fe9e00d1100 msgr2=0x7fe9e01588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 shutdown_connections 2024-01-19T04:55:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:55.994+0000 7fe9e6bb0640 1 -- 172.21.15.31:0/2472581251 wait complete. 2024-01-19T04:55:56.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:56.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.108+0000 7f8e06082640 1 Processor -- start 2024-01-19T04:55:56.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.108+0000 7f8e06082640 1 -- start start 2024-01-19T04:55:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.108+0000 7f8e06082640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 0x7f8e000d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.108+0000 7f8e06082640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e000cce50 con 0x7f8e000cd2e0 2024-01-19T04:55:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.108+0000 7f8e06082640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e000cc240 con 0x7f8e0015b6b0 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.108+0000 7f8e06082640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000d03b0 con 0x7f8e000ce090 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8e000cd2e0 0x7f8e000b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55022/0 (socket says 172.21.15.31:55022) 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfffff640 1 -- 172.21.15.31:0/442781780 learned_addr learned my addr 172.21.15.31:0/442781780 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfeffd640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 0x7f8e000d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32391208 0 0) 0x7f8e000cce50 con 0x7f8e000cd2e0 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de8003520 con 0x7f8e000cd2e0 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850776271 0 0) 0x7f8e000cc240 con 0x7f8e0015b6b0 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e000cce50 con 0x7f8e0015b6b0 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2278415160 0 0) 0x7f8de8003520 con 0x7f8e000cd2e0 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 msgr2=0x7f8e000d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 0x7f8e000d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 >> v1:172.21.15.137:6789/0 conn(0x7f8e0015b6b0 legacy=0x7f8e000cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000cc520 con 0x7f8e000cd2e0 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f8df80019f0 con 0x7f8e000cd2e0 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfeffd640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 0x7f8e000d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 >> v1:172.21.15.31:6789/0 conn(0x7f8e000cd2e0 legacy=0x7f8e000b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8de8003b30 0x7f8de8003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8de80047c0 0x7f8de8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 0x7f8de8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000d03b0 con 0x7f8de8003b30 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de800ba30 con 0x7f8de80047c0 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de800bbc0 con 0x7f8e000ce090 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dff7fe640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8de8003b30 0x7f8de8003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.109+0000 7f8dfeffd640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8de80047c0 0x7f8de8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dfffff640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 0x7f8de8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dff7fe640 1 -- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 msgr2=0x7f8de8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dff7fe640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e000ce090 0x7f8de8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dff7fe640 1 -- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8de80047c0 msgr2=0x7f8de8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dff7fe640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8de80047c0 0x7f8de8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dff7fe640 1 -- 172.21.15.31:0/442781780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4009400 con 0x7f8de8003b30 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dff7fe640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8de8003b30 0x7f8de8003f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8df0007fd0 tx=0x7f8de80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282706ae13b3a4c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.110+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00052d0 con 0x7f8de8003b30 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df0005430 con 0x7f8de8003b30 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8dfe7fc640 1 -- 172.21.15.31:0/442781780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0003860 con 0x7f8de8003b30 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8e06082640 1 -- 172.21.15.31:0/442781780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8de8003b30 msgr2=0x7f8de8003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8e06082640 1 --2- 172.21.15.31:0/442781780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8de8003b30 0x7f8de8003f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8df0007fd0 tx=0x7f8de80057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8dfffff640 1 -- 172.21.15.31:0/442781780 reap_dead start 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8e06082640 1 -- 172.21.15.31:0/442781780 shutdown_connections 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8e06082640 1 -- 172.21.15.31:0/442781780 >> 172.21.15.31:0/442781780 conn(0x7f8e000d1100 msgr2=0x7f8e000d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8e06082640 1 -- 172.21.15.31:0/442781780 shutdown_connections 2024-01-19T04:55:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.111+0000 7f8e06082640 1 -- 172.21.15.31:0/442781780 wait complete. 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.112+0000 7f8e06082640 1 Processor -- start 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.112+0000 7f8e06082640 1 -- start start 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8e06082640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8de8003b30 0x7f8e001571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8e06082640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000b4df0 0x7f8e001f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8e06082640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e000c6660 0x7f8e000ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8e06082640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001f4e60 con 0x7f8e000b4df0 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8e06082640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000b5250 con 0x7f8e000c6660 2024-01-19T04:55:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8e06082640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001f57c0 con 0x7f8de8003b30 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000b4df0 0x7f8e001f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000b4df0 0x7f8e001f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48500/0 (socket says 172.21.15.31:48500) 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8dfeffd640 1 -- 172.21.15.31:0/3525112462 learned_addr learned my addr 172.21.15.31:0/3525112462 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8dff7fe640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8de8003b30 0x7f8e001571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8dfffff640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e000c6660 0x7f8e000ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.113+0000 7f8dfeffd640 1 -- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8de8003b30 msgr2=0x7f8e001571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8dfeffd640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8de8003b30 0x7f8e001571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8dfeffd640 1 -- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e000c6660 msgr2=0x7f8e000ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8dfeffd640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e000c6660 0x7f8e000ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8dfeffd640 1 -- 172.21.15.31:0/3525112462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e001f4830 con 0x7f8e000b4df0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8dfeffd640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000b4df0 0x7f8e001f5280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8df4011d30 tx=0x7f8df400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8de7fff640 1 -- 172.21.15.31:0/3525112462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df400fcc0 con 0x7f8e000b4df0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8de7fff640 1 -- 172.21.15.31:0/3525112462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df400fe20 con 0x7f8e000b4df0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8de7fff640 1 -- 172.21.15.31:0/3525112462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004240 con 0x7f8e000b4df0 2024-01-19T04:55:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e000b5480 con 0x7f8e000b4df0 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.114+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e001f4a80 con 0x7f8e000b4df0 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.115+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e00074900 con 0x7f8e000b4df0 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.115+0000 7f8de7fff640 1 -- 172.21.15.31:0/3525112462 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8df40043a0 con 0x7f8e000b4df0 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.116+0000 7f8de7fff640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8dec06aed0 0x7f8dec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.116+0000 7f8de7fff640 1 -- 172.21.15.31:0/3525112462 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8df4089db0 con 0x7f8e000b4df0 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.116+0000 7f8dff7fe640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8dec06aed0 0x7f8dec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.116+0000 7f8dff7fe640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8dec06aed0 0x7f8dec06d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8df0034000 tx=0x7f8df0031000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.120+0000 7f8de7fff640 1 -- 172.21.15.31:0/3525112462 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8df4054c70 con 0x7f8e000b4df0 2024-01-19T04:55:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.296+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e000cf4c0 con 0x7f8e000b4df0 2024-01-19T04:55:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.296+0000 7f8de7fff640 1 -- 172.21.15.31:0/3525112462 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8df4022020 con 0x7f8e000b4df0 2024-01-19T04:55:56.296 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8dec06aed0 msgr2=0x7f8dec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8dec06aed0 0x7f8dec06d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8df0034000 tx=0x7f8df0031000 comp rx=0 tx=0).stop 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000b4df0 msgr2=0x7f8e001f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000b4df0 0x7f8e001f5280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8df4011d30 tx=0x7f8df400f610 comp rx=0 tx=0).stop 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 shutdown_connections 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8dec06aed0 0x7f8dec06d390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e000c6660 0x7f8e000ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000b4df0 0x7f8e001f5280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 --2- 172.21.15.31:0/3525112462 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8de8003b30 0x7f8e001571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 >> 172.21.15.31:0/3525112462 conn(0x7f8e000d1100 msgr2=0x7f8e0015d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 shutdown_connections 2024-01-19T04:55:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.298+0000 7f8e06082640 1 -- 172.21.15.31:0/3525112462 wait complete. 2024-01-19T04:55:56.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:56.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.413+0000 7fe142e4f640 1 Processor -- start 2024-01-19T04:55:56.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.413+0000 7fe142e4f640 1 -- start start 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe142e4f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c15d700 0x7fe13c0d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe142e4f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe13c0d1fe0 0x7fe13c0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe142e4f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c0cef10 con 0x7fe13c0d11b0 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe142e4f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c0ce300 con 0x7fe13c0d1fe0 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe142e4f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0ce5e0 con 0x7fe13c15d700 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe1413ec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe13c0d1fe0 0x7fe13c0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe1413ec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe13c0d1fe0 0x7fe13c0b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48504/0 (socket says 172.21.15.31:48504) 2024-01-19T04:55:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe1413ec640 1 -- 172.21.15.31:0/2569571492 learned_addr learned my addr 172.21.15.31:0/2569571492 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe140beb640 1 --2- 172.21.15.31:0/2569571492 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c15d700 0x7fe13c0d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe1413ec640 1 -- 172.21.15.31:0/2569571492 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c15d700 msgr2=0x7fe13c0d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe1413ec640 1 --2- 172.21.15.31:0/2569571492 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c15d700 0x7fe13c0d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe1413ec640 1 -- 172.21.15.31:0/2569571492 >> v1:172.21.15.137:6789/0 conn(0x7fe13c0d11b0 legacy=0x7fe13c0d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.414+0000 7fe1413ec640 1 -- 172.21.15.31:0/2569571492 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0ce8c0 con 0x7fe13c0d1fe0 2024-01-19T04:55:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.415+0000 7fe1413ec640 1 --2- 172.21.15.31:0/2569571492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b2890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe134009e50 tx=0x7fe13402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1419faa29ac3f0be server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.415+0000 7fe13b7fe640 1 -- 172.21.15.31:0/2569571492 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13402cb00 con 0x7fe13c0d1fe0 2024-01-19T04:55:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.415+0000 7fe13b7fe640 1 -- 172.21.15.31:0/2569571492 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13402cc60 con 0x7fe13c0d1fe0 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.415+0000 7fe13b7fe640 1 -- 172.21.15.31:0/2569571492 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1340343f0 con 0x7fe13c0d1fe0 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 -- 172.21.15.31:0/2569571492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c0d1fe0 msgr2=0x7fe13c0b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2569571492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b2890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe134009e50 tx=0x7fe13402bf60 comp rx=0 tx=0).stop 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 -- 172.21.15.31:0/2569571492 shutdown_connections 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2569571492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b2890 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2569571492 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c15d700 0x7fe13c0d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 -- 172.21.15.31:0/2569571492 >> 172.21.15.31:0/2569571492 conn(0x7fe13c0d32f0 msgr2=0x7fe13c0d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 -- 172.21.15.31:0/2569571492 shutdown_connections 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.416+0000 7fe142e4f640 1 -- 172.21.15.31:0/2569571492 wait complete. 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.417+0000 7fe142e4f640 1 Processor -- start 2024-01-19T04:55:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.417+0000 7fe142e4f640 1 -- start start 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe142e4f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c0d11b0 0x7fe13c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe142e4f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe142e4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c15d700 0x7fe13c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe142e4f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c15ebc0 con 0x7fe13c15d700 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe142e4f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c156bd0 con 0x7fe13c0d1fe0 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe142e4f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c15df60 con 0x7fe13c0d11b0 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c15d700 0x7fe13c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c15d700 0x7fe13c0b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48510/0 (socket says 172.21.15.31:48510) 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 -- 172.21.15.31:0/2277130270 learned_addr learned my addr 172.21.15.31:0/2277130270 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:56.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe140beb640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c0d11b0 0x7fe13c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe13bfff640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 -- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c0d11b0 msgr2=0x7fe13c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c0d11b0 0x7fe13c156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 -- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe13c0d1fe0 msgr2=0x7fe13c0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe1413ec640 1 -- 172.21.15.31:0/2277130270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe134009b30 con 0x7fe13c15d700 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe140beb640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c0d11b0 0x7fe13c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.418+0000 7fe13bfff640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.419+0000 7fe1413ec640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c15d700 0x7fe13c0b1b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe134036040 tx=0x7fe134004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.419+0000 7fe1397fa640 1 -- 172.21.15.31:0/2277130270 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134009510 con 0x7fe13c15d700 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.419+0000 7fe1397fa640 1 -- 172.21.15.31:0/2277130270 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13403e450 con 0x7fe13c15d700 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.419+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c15e220 con 0x7fe13c15d700 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.419+0000 7fe1397fa640 1 -- 172.21.15.31:0/2277130270 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13403e7e0 con 0x7fe13c15d700 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.419+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c15e7e0 con 0x7fe13c15d700 2024-01-19T04:55:56.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.420+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c15ed00 con 0x7fe13c15d700 2024-01-19T04:55:56.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.420+0000 7fe1397fa640 1 -- 172.21.15.31:0/2277130270 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe13403e940 con 0x7fe13c15d700 2024-01-19T04:55:56.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.421+0000 7fe1397fa640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe12806aed0 0x7fe12806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.421+0000 7fe1397fa640 1 -- 172.21.15.31:0/2277130270 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1340ac5f0 con 0x7fe13c15d700 2024-01-19T04:55:56.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.421+0000 7fe140beb640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe12806aed0 0x7fe12806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.421+0000 7fe140beb640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe12806aed0 0x7fe12806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe12c006fd0 tx=0x7fe12c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.425+0000 7fe1397fa640 1 -- 172.21.15.31:0/2277130270 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe13c15ed00 con 0x7fe13c15d700 2024-01-19T04:55:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.606+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe13c15ed00 con 0x7fe13c15d700 2024-01-19T04:55:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.606+0000 7fe1397fa640 1 -- 172.21.15.31:0/2277130270 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe13c15ed00 con 0x7fe13c15d700 2024-01-19T04:55:56.606 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe12806aed0 msgr2=0x7fe12806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe12806aed0 0x7fe12806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe12c006fd0 tx=0x7fe12c008040 comp rx=0 tx=0).stop 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c15d700 msgr2=0x7fe13c0b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c15d700 0x7fe13c0b1b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe134036040 tx=0x7fe134004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 shutdown_connections 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe12806aed0 0x7fe12806d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe13c15d700 0x7fe13c0b1b30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe13c0d1fe0 0x7fe13c0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 --2- 172.21.15.31:0/2277130270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe13c0d11b0 0x7fe13c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 >> 172.21.15.31:0/2277130270 conn(0x7fe13c0d32f0 msgr2=0x7fe13c06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:56.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 shutdown_connections 2024-01-19T04:55:56.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.608+0000 7fe142e4f640 1 -- 172.21.15.31:0/2277130270 wait complete. 2024-01-19T04:55:56.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:55:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.721+0000 7fbd7bc61640 1 Processor -- start 2024-01-19T04:55:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.721+0000 7fbd7bc61640 1 -- start start 2024-01-19T04:55:56.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd7bc61640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd7415b230 0x7fbd740cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd7bc61640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b610 0x7fbd740ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd7bc61640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd740ceeb0 con 0x7fbd740cdb50 2024-01-19T04:55:56.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd7bc61640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd740cf190 con 0x7fbd7415b230 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd7bc61640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740cf470 con 0x7fbd7415b610 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd799fd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd7415b230 0x7fbd740cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd799fd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd7415b230 0x7fbd740cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48520/0 (socket says 172.21.15.31:48520) 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd799fd640 1 -- 172.21.15.31:0/2562240413 learned_addr learned my addr 172.21.15.31:0/2562240413 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.722+0000 7fbd791fc640 1 --2- 172.21.15.31:0/2562240413 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b610 0x7fbd740ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.723+0000 7fbd799fd640 1 -- 172.21.15.31:0/2562240413 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b610 msgr2=0x7fbd740ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.723+0000 7fbd799fd640 1 --2- 172.21.15.31:0/2562240413 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b610 0x7fbd740ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.723+0000 7fbd799fd640 1 -- 172.21.15.31:0/2562240413 >> v1:172.21.15.137:6789/0 conn(0x7fbd740cdb50 legacy=0x7fbd7415efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.723+0000 7fbd799fd640 1 -- 172.21.15.31:0/2562240413 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd740cf750 con 0x7fbd7415b230 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.723+0000 7fbd799fd640 1 --2- 172.21.15.31:0/2562240413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b230 0x7fbd740cc7d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009e50 tx=0x7fbd7002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac985609d83fdd4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.723+0000 7fbd789fb640 1 -- 172.21.15.31:0/2562240413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7002cb00 con 0x7fbd7415b230 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd789fb640 1 -- 172.21.15.31:0/2562240413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7002cc60 con 0x7fbd7415b230 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd789fb640 1 -- 172.21.15.31:0/2562240413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd700343f0 con 0x7fbd7415b230 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 -- 172.21.15.31:0/2562240413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b230 msgr2=0x7fbd740cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/2562240413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b230 0x7fbd740cc7d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009e50 tx=0x7fbd7002bf60 comp rx=0 tx=0).stop 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 -- 172.21.15.31:0/2562240413 shutdown_connections 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/2562240413 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b610 0x7fbd740ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/2562240413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b230 0x7fbd740cc7d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 -- 172.21.15.31:0/2562240413 >> 172.21.15.31:0/2562240413 conn(0x7fbd7408afe0 msgr2=0x7fbd740d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 -- 172.21.15.31:0/2562240413 shutdown_connections 2024-01-19T04:55:56.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.724+0000 7fbd7bc61640 1 -- 172.21.15.31:0/2562240413 wait complete. 2024-01-19T04:55:56.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.725+0000 7fbd7bc61640 1 Processor -- start 2024-01-19T04:55:56.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.725+0000 7fbd7bc61640 1 -- start start 2024-01-19T04:55:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7bc61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd740cdb50 0x7fbd740c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7bc61640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b230 0x7fbd7415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7bc61640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b610 0x7fbd740c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7bc61640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740c70b0 con 0x7fbd7415b610 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7bc61640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740c6200 con 0x7fbd740cdb50 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7bc61640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740c64a0 con 0x7fbd7415b230 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b610 0x7fbd740c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b610 0x7fbd740c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48522/0 (socket says 172.21.15.31:48522) 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 -- 172.21.15.31:0/1061590212 learned_addr learned my addr 172.21.15.31:0/1061590212 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd791fc640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b230 0x7fbd7415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 -- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b230 msgr2=0x7fbd7415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd799fd640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd740cdb50 0x7fbd740c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b230 0x7fbd7415a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 -- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd740cdb50 msgr2=0x7fbd740c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd740cdb50 0x7fbd740c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.726+0000 7fbd7a1fe640 1 -- 172.21.15.31:0/1061590212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70009b30 con 0x7fbd7415b610 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.727+0000 7fbd7a1fe640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b610 0x7fbd740c5cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c3c0 tx=0x7fbd6c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.727+0000 7fbd6a7fc640 1 -- 172.21.15.31:0/1061590212 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0052d0 con 0x7fbd7415b610 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.727+0000 7fbd6a7fc640 1 -- 172.21.15.31:0/1061590212 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c005450 con 0x7fbd7415b610 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.727+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd740d00d0 con 0x7fbd7415b610 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.727+0000 7fbd6a7fc640 1 -- 172.21.15.31:0/1061590212 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c007260 con 0x7fbd7415b610 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.727+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd740c6a60 con 0x7fbd7415b610 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.728+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740c7260 con 0x7fbd7415b610 2024-01-19T04:55:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.728+0000 7fbd6a7fc640 1 -- 172.21.15.31:0/1061590212 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fbd6c0073c0 con 0x7fbd7415b610 2024-01-19T04:55:56.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.729+0000 7fbd6a7fc640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbd5806af00 0x7fbd5806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:56.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.729+0000 7fbd6a7fc640 1 -- 172.21.15.31:0/1061590212 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd6c088900 con 0x7fbd7415b610 2024-01-19T04:55:56.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.729+0000 7fbd799fd640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbd5806af00 0x7fbd5806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:56.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.729+0000 7fbd799fd640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbd5806af00 0x7fbd5806d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd70035000 tx=0x7fbd70030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:56.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.733+0000 7fbd6a7fc640 1 -- 172.21.15.31:0/1061590212 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbd740c7260 con 0x7fbd7415b610 2024-01-19T04:55:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.921+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd74074900 con 0x7fbd7415b610 2024-01-19T04:55:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.921+0000 7fbd6a7fc640 1 -- 172.21.15.31:0/1061590212 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbd6c0537d0 con 0x7fbd7415b610 2024-01-19T04:55:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:55:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbd5806af00 msgr2=0x7fbd5806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbd5806af00 0x7fbd5806d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd70035000 tx=0x7fbd70030000 comp rx=0 tx=0).stop 2024-01-19T04:55:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b610 msgr2=0x7fbd740c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b610 0x7fbd740c5cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c3c0 tx=0x7fbd6c00c880 comp rx=0 tx=0).stop 2024-01-19T04:55:56.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 shutdown_connections 2024-01-19T04:55:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbd5806af00 0x7fbd5806d3c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd7415b610 0x7fbd740c5cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd7415b230 0x7fbd7415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 --2- 172.21.15.31:0/1061590212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd740cdb50 0x7fbd740c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 >> 172.21.15.31:0/1061590212 conn(0x7fbd7408afe0 msgr2=0x7fbd7415e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 shutdown_connections 2024-01-19T04:55:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:56.923+0000 7fbd7bc61640 1 -- 172.21.15.31:0/1061590212 wait complete. 2024-01-19T04:55:56.933 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:55:56.933 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:56.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.055+0000 7fe9b14e5640 1 Processor -- start 2024-01-19T04:55:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.055+0000 7fe9b14e5640 1 -- start start 2024-01-19T04:55:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9b14e5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac160bb0 0x7fe9ac0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9b14e5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9ac0cfd10 0x7fe9ac0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9b14e5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac0d06b0 con 0x7fe9ac0d0ef0 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9b14e5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b66b0 con 0x7fe9ac0cfd10 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9b14e5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0d1b90 con 0x7fe9ac160bb0 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9aa7fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9ac0cfd10 0x7fe9ac0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9aa7fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9ac0cfd10 0x7fe9ac0d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48532/0 (socket says 172.21.15.31:48532) 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1848163612 learned_addr learned my addr 172.21.15.31:0/1848163612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1848163612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac160bb0 msgr2=0x7fe9ac0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9aa7fc640 1 --2- 172.21.15.31:0/1848163612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac160bb0 0x7fe9ac0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1848163612 >> v1:172.21.15.137:6789/0 conn(0x7fe9ac0d0ef0 legacy=0x7fe9ac0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.056+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1848163612 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0d1e70 con 0x7fe9ac0cfd10 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9aa7fc640 1 --2- 172.21.15.31:0/1848163612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac0cfd10 0x7fe9ac0d0170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009870 tx=0x7fe9a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191dd50e86a65d1b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9a9ffb640 1 -- 172.21.15.31:0/1848163612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0031640 con 0x7fe9ac0cfd10 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9a9ffb640 1 -- 172.21.15.31:0/1848163612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a00317a0 con 0x7fe9ac0cfd10 2024-01-19T04:55:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9a9ffb640 1 -- 172.21.15.31:0/1848163612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0031b30 con 0x7fe9ac0cfd10 2024-01-19T04:55:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 -- 172.21.15.31:0/1848163612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac0cfd10 msgr2=0x7fe9ac0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/1848163612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac0cfd10 0x7fe9ac0d0170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009870 tx=0x7fe9a002b8f0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 -- 172.21.15.31:0/1848163612 shutdown_connections 2024-01-19T04:55:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/1848163612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac0cfd10 0x7fe9ac0d0170 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/1848163612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac160bb0 0x7fe9ac0b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 -- 172.21.15.31:0/1848163612 >> 172.21.15.31:0/1848163612 conn(0x7fe9ac09c600 msgr2=0x7fe9ac0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 -- 172.21.15.31:0/1848163612 shutdown_connections 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.057+0000 7fe9b14e5640 1 -- 172.21.15.31:0/1848163612 wait complete. 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 Processor -- start 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 -- start start 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac0cfd10 0x7fe9ac1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ac0d0ef0 0x7fe9ac0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac160bb0 0x7fe9ac0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b3da0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0cac40 con 0x7fe9ac0d0ef0 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.059+0000 7fe9b14e5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0caf20 con 0x7fe9ac0cfd10 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac160bb0 0x7fe9ac0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac160bb0 0x7fe9ac0c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48542/0 (socket says 172.21.15.31:48542) 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 -- 172.21.15.31:0/4157360566 learned_addr learned my addr 172.21.15.31:0/4157360566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9aa7fc640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ac0d0ef0 0x7fe9ac0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9aaffd640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac0cfd10 0x7fe9ac1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 -- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac0cfd10 msgr2=0x7fe9ac1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac0cfd10 0x7fe9ac1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 -- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ac0d0ef0 msgr2=0x7fe9ac0ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ac0d0ef0 0x7fe9ac0ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 -- 172.21.15.31:0/4157360566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00094d0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9ab7fe640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac160bb0 0x7fe9ac0c9460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400b990 tx=0x7fe9a400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9937fe640 1 -- 172.21.15.31:0/4157360566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400caa0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9937fe640 1 -- 172.21.15.31:0/4157360566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a400cc00 con 0x7fe9ac160bb0 2024-01-19T04:55:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9937fe640 1 -- 172.21.15.31:0/4157360566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4014440 con 0x7fe9ac160bb0 2024-01-19T04:55:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac0cb200 con 0x7fe9ac160bb0 2024-01-19T04:55:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.060+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac0cb7c0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.061+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac159740 con 0x7fe9ac160bb0 2024-01-19T04:55:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.062+0000 7fe9937fe640 1 -- 172.21.15.31:0/4157360566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe9a401d3b0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.062+0000 7fe9937fe640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe99806aed0 0x7fe99806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.062+0000 7fe9937fe640 1 -- 172.21.15.31:0/4157360566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9a4089420 con 0x7fe9ac160bb0 2024-01-19T04:55:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.062+0000 7fe9aaffd640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe99806aed0 0x7fe99806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.063+0000 7fe9aaffd640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe99806aed0 0x7fe99806d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009990 tx=0x7fe99c009110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.066+0000 7fe9937fe640 1 -- 172.21.15.31:0/4157360566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe9a40543b0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.244+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9ac0696e0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.245+0000 7fe9937fe640 1 -- 172.21.15.31:0/4157360566 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fe9a40579d0 con 0x7fe9ac160bb0 2024-01-19T04:55:57.245 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:57.245 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe99806aed0 msgr2=0x7fe99806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe99806aed0 0x7fe99806d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009990 tx=0x7fe99c009110 comp rx=0 tx=0).stop 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac160bb0 msgr2=0x7fe9ac0c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac160bb0 0x7fe9ac0c9460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400b990 tx=0x7fe9a400bd90 comp rx=0 tx=0).stop 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 shutdown_connections 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe99806aed0 0x7fe99806d390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ac160bb0 0x7fe9ac0c9460 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ac0d0ef0 0x7fe9ac0ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 --2- 172.21.15.31:0/4157360566 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9ac0cfd10 0x7fe9ac1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 >> 172.21.15.31:0/4157360566 conn(0x7fe9ac09c600 msgr2=0x7fe9ac0d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 shutdown_connections 2024-01-19T04:55:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.248+0000 7fe9b14e5640 1 -- 172.21.15.31:0/4157360566 wait complete. 2024-01-19T04:55:57.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:57.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.368+0000 7fcc77d3a640 1 Processor -- start 2024-01-19T04:55:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.369+0000 7fcc77d3a640 1 -- start start 2024-01-19T04:55:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.369+0000 7fcc77d3a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcc700cb2e0 0x7fcc7015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.369+0000 7fcc77d3a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcc700716f0 0x7fcc7015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.369+0000 7fcc77d3a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7015ece0 con 0x7fcc7015b510 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.369+0000 7fcc77d3a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc70071c20 con 0x7fcc700cb2e0 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.369+0000 7fcc77d3a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc70071f00 con 0x7fcc700716f0 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc75ad6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcc700cb2e0 0x7fcc7015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc762d7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc7015b510 0x7fcc7006d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55036/0 (socket says 172.21.15.31:55036) 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc762d7640 1 -- 172.21.15.31:0/848373409 learned_addr learned my addr 172.21.15.31:0/848373409 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc752d5640 1 --2- 172.21.15.31:0/848373409 >> v2:172.21.15.137:3300/0 conn(0x7fcc700716f0 0x7fcc7015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc75ad6640 1 -- 172.21.15.31:0/848373409 >> v2:172.21.15.137:3300/0 conn(0x7fcc700716f0 msgr2=0x7fcc7015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc75ad6640 1 --2- 172.21.15.31:0/848373409 >> v2:172.21.15.137:3300/0 conn(0x7fcc700716f0 0x7fcc7015afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc75ad6640 1 -- 172.21.15.31:0/848373409 >> v1:172.21.15.31:6789/0 conn(0x7fcc7015b510 legacy=0x7fcc7006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc74ad4640 1 -- 172.21.15.31:0/848373409 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097223011 0 0) 0x7fcc7015ece0 con 0x7fcc7015b510 2024-01-19T04:55:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.370+0000 7fcc75ad6640 1 -- 172.21.15.31:0/848373409 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc700721e0 con 0x7fcc700cb2e0 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc75ad6640 1 --2- 172.21.15.31:0/848373409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700cb2e0 0x7fcc7015e7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc700cff30 tx=0x7fcc6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44677e38bbbce8d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc74ad4640 1 -- 172.21.15.31:0/848373409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c031830 con 0x7fcc700cb2e0 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc74ad4640 1 -- 172.21.15.31:0/848373409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c031990 con 0x7fcc700cb2e0 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc74ad4640 1 -- 172.21.15.31:0/848373409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c031d20 con 0x7fcc700cb2e0 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc77d3a640 1 -- 172.21.15.31:0/848373409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700cb2e0 msgr2=0x7fcc7015e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/848373409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700cb2e0 0x7fcc7015e7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc700cff30 tx=0x7fcc6c02b8d0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc77d3a640 1 -- 172.21.15.31:0/848373409 shutdown_connections 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/848373409 >> v2:172.21.15.137:3300/0 conn(0x7fcc700716f0 0x7fcc7015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/848373409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700cb2e0 0x7fcc7015e7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.371+0000 7fcc77d3a640 1 -- 172.21.15.31:0/848373409 >> 172.21.15.31:0/848373409 conn(0x7fcc7006aa00 msgr2=0x7fcc7006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.372+0000 7fcc77d3a640 1 -- 172.21.15.31:0/848373409 shutdown_connections 2024-01-19T04:55:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.372+0000 7fcc77d3a640 1 -- 172.21.15.31:0/848373409 wait complete. 2024-01-19T04:55:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 Processor -- start 2024-01-19T04:55:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 -- start start 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700716f0 0x7fcc7014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcc700cb2e0 0x7fcc70150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc7015b510 0x7fcc7014d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7014e510 con 0x7fcc700716f0 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7014d430 con 0x7fcc7015b510 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.373+0000 7fcc77d3a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc70151400 con 0x7fcc700cb2e0 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700716f0 0x7fcc7014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700716f0 0x7fcc7014c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48552/0 (socket says 172.21.15.31:48552) 2024-01-19T04:55:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 -- 172.21.15.31:0/3206387392 learned_addr learned my addr 172.21.15.31:0/3206387392 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc752d5640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcc700cb2e0 0x7fcc70150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 -- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcc700cb2e0 msgr2=0x7fcc70150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcc700cb2e0 0x7fcc70150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 -- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc7015b510 msgr2=0x7fcc7014d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc7015b510 0x7fcc7014d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 -- 172.21.15.31:0/3206387392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c009400 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc75ad6640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700716f0 0x7fcc7014c1d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009890 tx=0x7fcc6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc667fc640 1 -- 172.21.15.31:0/3206387392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c03ba70 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc667fc640 1 -- 172.21.15.31:0/3206387392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c03bbd0 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7014c7b0 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc667fc640 1 -- 172.21.15.31:0/3206387392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0453f0 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.374+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7014cd70 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.375+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7015a2e0 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.376+0000 7fcc667fc640 1 -- 172.21.15.31:0/3206387392 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fcc6c045550 con 0x7fcc700716f0 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.376+0000 7fcc667fc640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcc5406aed0 0x7fcc5406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.376+0000 7fcc667fc640 1 -- 172.21.15.31:0/3206387392 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc6c0ab8e0 con 0x7fcc700716f0 2024-01-19T04:55:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.376+0000 7fcc752d5640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcc5406aed0 0x7fcc5406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.377+0000 7fcc752d5640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcc5406aed0 0x7fcc5406d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006fd0 tx=0x7fcc60008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.380+0000 7fcc667fc640 1 -- 172.21.15.31:0/3206387392 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fcc6c0768d0 con 0x7fcc700716f0 2024-01-19T04:55:57.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.558+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc700cf620 con 0x7fcc700716f0 2024-01-19T04:55:57.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.558+0000 7fcc667fc640 1 -- 172.21.15.31:0/3206387392 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc6c0350a0 con 0x7fcc700716f0 2024-01-19T04:55:57.558 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcc5406aed0 msgr2=0x7fcc5406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcc5406aed0 0x7fcc5406d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006fd0 tx=0x7fcc60008040 comp rx=0 tx=0).stop 2024-01-19T04:55:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700716f0 msgr2=0x7fcc7014c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700716f0 0x7fcc7014c1d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009890 tx=0x7fcc6c0046c0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 shutdown_connections 2024-01-19T04:55:57.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fcc5406aed0 0x7fcc5406d390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc7015b510 0x7fcc7014d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcc700cb2e0 0x7fcc70150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 --2- 172.21.15.31:0/3206387392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc700716f0 0x7fcc7014c1d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 >> 172.21.15.31:0/3206387392 conn(0x7fcc7006aa00 msgr2=0x7fcc7015e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:57.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 shutdown_connections 2024-01-19T04:55:57.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.560+0000 7fcc77d3a640 1 -- 172.21.15.31:0/3206387392 wait complete. 2024-01-19T04:55:57.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:57.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.675+0000 7fd87c9be640 1 Processor -- start 2024-01-19T04:55:57.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.675+0000 7fd87c9be640 1 -- start start 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.676+0000 7fd87c9be640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd87815d720 0x7fd8780d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.676+0000 7fd87c9be640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8780d3ea0 0x7fd8780d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.676+0000 7fd87c9be640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8780d1a50 0x7fd8780cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.676+0000 7fd87c9be640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8780d1f00 con 0x7fd8780d1a50 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.676+0000 7fd87c9be640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8780cf160 con 0x7fd87815d720 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.676+0000 7fd87c9be640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780ce8c0 con 0x7fd8780d3ea0 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8780d1a50 0x7fd8780cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8780d1a50 0x7fd8780cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48568/0 (socket says 172.21.15.31:48568) 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 -- 172.21.15.31:0/3298256554 learned_addr learned my addr 172.21.15.31:0/3298256554 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 -- 172.21.15.31:0/3298256554 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8780d3ea0 msgr2=0x7fd8780d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd875d9b640 1 --2- 172.21.15.31:0/3298256554 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8780d3ea0 0x7fd8780d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd87659c640 1 --2- 172.21.15.31:0/3298256554 >> v2:172.21.15.137:3300/0 conn(0x7fd87815d720 0x7fd8780d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 --2- 172.21.15.31:0/3298256554 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8780d3ea0 0x7fd8780d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 -- 172.21.15.31:0/3298256554 >> v2:172.21.15.137:3300/0 conn(0x7fd87815d720 msgr2=0x7fd8780d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 --2- 172.21.15.31:0/3298256554 >> v2:172.21.15.137:3300/0 conn(0x7fd87815d720 0x7fd8780d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd876d9d640 1 -- 172.21.15.31:0/3298256554 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780ceba0 con 0x7fd8780d1a50 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd875d9b640 1 --2- 172.21.15.31:0/3298256554 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8780d3ea0 0x7fd8780d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.677+0000 7fd87659c640 1 --2- 172.21.15.31:0/3298256554 >> v2:172.21.15.137:3300/0 conn(0x7fd87815d720 0x7fd8780d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd876d9d640 1 --2- 172.21.15.31:0/3298256554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d1a50 0x7fd8780cfc70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd878067310 tx=0x7fd86802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0131f3d2ea170d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87559a640 1 -- 172.21.15.31:0/3298256554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86802c950 con 0x7fd8780d1a50 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87559a640 1 -- 172.21.15.31:0/3298256554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86802cab0 con 0x7fd8780d1a50 2024-01-19T04:55:57.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87559a640 1 -- 172.21.15.31:0/3298256554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86802ce40 con 0x7fd8780d1a50 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 -- 172.21.15.31:0/3298256554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d1a50 msgr2=0x7fd8780cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 --2- 172.21.15.31:0/3298256554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d1a50 0x7fd8780cfc70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd878067310 tx=0x7fd86802bdb0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 -- 172.21.15.31:0/3298256554 shutdown_connections 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 --2- 172.21.15.31:0/3298256554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d1a50 0x7fd8780cfc70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 --2- 172.21.15.31:0/3298256554 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8780d3ea0 0x7fd8780d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 --2- 172.21.15.31:0/3298256554 >> v2:172.21.15.137:3300/0 conn(0x7fd87815d720 0x7fd8780d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 -- 172.21.15.31:0/3298256554 >> 172.21.15.31:0/3298256554 conn(0x7fd8780d3310 msgr2=0x7fd8780d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 -- 172.21.15.31:0/3298256554 shutdown_connections 2024-01-19T04:55:57.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.678+0000 7fd87c9be640 1 -- 172.21.15.31:0/3298256554 wait complete. 2024-01-19T04:55:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.680+0000 7fd87c9be640 1 Processor -- start 2024-01-19T04:55:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.680+0000 7fd87c9be640 1 -- start start 2024-01-19T04:55:57.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd87c9be640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8780d1a50 0x7fd87815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd87c9be640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d3ea0 0x7fd8780b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd87c9be640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd87815d720 0x7fd8780cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd87c9be640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87815efd0 con 0x7fd8780d3ea0 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd87c9be640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780b4e90 con 0x7fd8780d1a50 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd87c9be640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87815e3c0 con 0x7fd87815d720 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d3ea0 0x7fd8780b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d3ea0 0x7fd8780b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48580/0 (socket says 172.21.15.31:48580) 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 -- 172.21.15.31:0/2361772756 learned_addr learned my addr 172.21.15.31:0/2361772756 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd876d9d640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd87815d720 0x7fd8780cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd87659c640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8780d1a50 0x7fd87815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 -- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd87815d720 msgr2=0x7fd8780cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd87815d720 0x7fd8780cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 -- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8780d1a50 msgr2=0x7fd87815dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8780d1a50 0x7fd87815dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 -- 172.21.15.31:0/2361772756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009b30 con 0x7fd8780d3ea0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd875d9b640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d3ea0 0x7fd8780b4950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c7c0 tx=0x7fd86000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.681+0000 7fd86effd640 1 -- 172.21.15.31:0/2361772756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860007820 con 0x7fd8780d3ea0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.682+0000 7fd86effd640 1 -- 172.21.15.31:0/2361772756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd860007980 con 0x7fd8780d3ea0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.682+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87815e6a0 con 0x7fd8780d3ea0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.682+0000 7fd86effd640 1 -- 172.21.15.31:0/2361772756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860007d10 con 0x7fd8780d3ea0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.682+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87815ec60 con 0x7fd8780d3ea0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.683+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87815f140 con 0x7fd8780d3ea0 2024-01-19T04:55:57.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.683+0000 7fd86effd640 1 -- 172.21.15.31:0/2361772756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd8600113f0 con 0x7fd8780d3ea0 2024-01-19T04:55:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.683+0000 7fd86effd640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd85806aed0 0x7fd85806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.684+0000 7fd86effd640 1 -- 172.21.15.31:0/2361772756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd860089740 con 0x7fd8780d3ea0 2024-01-19T04:55:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.684+0000 7fd87659c640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd85806aed0 0x7fd85806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.684+0000 7fd87659c640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd85806aed0 0x7fd85806d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd87000aac0 tx=0x7fd870008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.688+0000 7fd86effd640 1 -- 172.21.15.31:0/2361772756 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd860054630 con 0x7fd8780d3ea0 2024-01-19T04:55:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.868+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd878074900 con 0x7fd8780d3ea0 2024-01-19T04:55:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.868+0000 7fd86effd640 1 -- 172.21.15.31:0/2361772756 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd86001f020 con 0x7fd8780d3ea0 2024-01-19T04:55:57.869 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd85806aed0 msgr2=0x7fd85806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd85806aed0 0x7fd85806d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd87000aac0 tx=0x7fd870008040 comp rx=0 tx=0).stop 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d3ea0 msgr2=0x7fd8780b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d3ea0 0x7fd8780b4950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c7c0 tx=0x7fd86000cc80 comp rx=0 tx=0).stop 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 shutdown_connections 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd85806aed0 0x7fd85806d390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd87815d720 0x7fd8780cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8780d3ea0 0x7fd8780b4950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 --2- 172.21.15.31:0/2361772756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8780d1a50 0x7fd87815dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 >> 172.21.15.31:0/2361772756 conn(0x7fd8780d3310 msgr2=0x7fd8780b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 shutdown_connections 2024-01-19T04:55:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.871+0000 7fd87c9be640 1 -- 172.21.15.31:0/2361772756 wait complete. 2024-01-19T04:55:57.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:57.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.987+0000 7fbdee91d640 1 Processor -- start 2024-01-19T04:55:57.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.987+0000 7fbdee91d640 1 -- start start 2024-01-19T04:55:57.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbdee91d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 0x7fbde80cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbdee91d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde80d1450 con 0x7fbde80d3f10 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbdee91d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde80cf240 con 0x7fbde815d5c0 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbdee91d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80ce6c0 con 0x7fbde80d2010 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbde77fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbde80d3f10 0x7fbde80d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55046/0 (socket says 172.21.15.31:55046) 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbde77fe640 1 -- 172.21.15.31:0/2995742667 learned_addr learned my addr 172.21.15.31:0/2995742667 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbdeceba640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 0x7fbde80cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536112999 0 0) 0x7fbde80d1450 con 0x7fbde80d3f10 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.988+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd0003520 con 0x7fbde80d3f10 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543848584 0 0) 0x7fbdd0003520 con 0x7fbde80d3f10 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 msgr2=0x7fbde80cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 0x7fbde80cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 >> v1:172.21.15.137:6789/0 conn(0x7fbde815d5c0 legacy=0x7fbde80cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80ce9a0 con 0x7fbde80d3f10 2024-01-19T04:55:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fbddc001670 con 0x7fbde80d3f10 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 >> v1:172.21.15.31:6789/0 conn(0x7fbde80d3f10 legacy=0x7fbde80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd00039a0 0x7fbdd0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 0x7fbdd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdd0004d00 0x7fbdd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80ce6c0 con 0x7fbdd00039a0 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd000ba30 con 0x7fbdd0004d00 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd000bba0 con 0x7fbde80d2010 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde7fff640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd00039a0 0x7fbdd0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbdeceba640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdd0004d00 0x7fbdd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde77fe640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 0x7fbdd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde7fff640 1 -- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 msgr2=0x7fbdd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde7fff640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbde80d2010 0x7fbdd00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde7fff640 1 -- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdd0004d00 msgr2=0x7fbdd0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde7fff640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdd0004d00 0x7fbdd0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.989+0000 7fbde7fff640 1 -- 172.21.15.31:0/2995742667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd0003520 con 0x7fbdd00039a0 2024-01-19T04:55:57.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.990+0000 7fbde7fff640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd00039a0 0x7fbdd0003da0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8001860 tx=0x7fbdd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e6a3226e2836d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.990+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8033630 con 0x7fbdd00039a0 2024-01-19T04:55:57.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.990+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80337b0 con 0x7fbdd00039a0 2024-01-19T04:55:57.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.990+0000 7fbde6ffd640 1 -- 172.21.15.31:0/2995742667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8033b60 con 0x7fbdd00039a0 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.991+0000 7fbdee91d640 1 -- 172.21.15.31:0/2995742667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd00039a0 msgr2=0x7fbdd0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.991+0000 7fbdee91d640 1 --2- 172.21.15.31:0/2995742667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd00039a0 0x7fbdd0003da0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8001860 tx=0x7fbdd00057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.991+0000 7fbdeceba640 1 -- 172.21.15.31:0/2995742667 reap_dead start 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.991+0000 7fbdee91d640 1 -- 172.21.15.31:0/2995742667 shutdown_connections 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.991+0000 7fbdee91d640 1 -- 172.21.15.31:0/2995742667 >> 172.21.15.31:0/2995742667 conn(0x7fbde809c600 msgr2=0x7fbde80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.991+0000 7fbdee91d640 1 -- 172.21.15.31:0/2995742667 shutdown_connections 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.991+0000 7fbdee91d640 1 -- 172.21.15.31:0/2995742667 wait complete. 2024-01-19T04:55:57.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.992+0000 7fbdee91d640 1 Processor -- start 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.992+0000 7fbdee91d640 1 -- start start 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbdee91d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbdd00039a0 0x7fbde80b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbdee91d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbde80b5f10 0x7fbde8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbdee91d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbde81603b0 0x7fbde8160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbdee91d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde815bc10 con 0x7fbde80b5f10 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbdee91d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8151c70 con 0x7fbde81603b0 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbdee91d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80b6370 con 0x7fbdd00039a0 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbde80b5f10 0x7fbde8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbde80b5f10 0x7fbde8151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48590/0 (socket says 172.21.15.31:48590) 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 -- 172.21.15.31:0/3341248158 learned_addr learned my addr 172.21.15.31:0/3341248158 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde7fff640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbdd00039a0 0x7fbde80b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 -- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbdd00039a0 msgr2=0x7fbde80b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbdd00039a0 0x7fbde80b74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbdeceba640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbde81603b0 0x7fbde8160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 -- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbde81603b0 msgr2=0x7fbde8160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbde81603b0 0x7fbde8160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 -- 172.21.15.31:0/3341248158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde815f310 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.993+0000 7fbde77fe640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbde80b5f10 0x7fbde8151730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0072c0 tx=0x7fbddc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.994+0000 7fbde4ff9640 1 -- 172.21.15.31:0/3341248158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00f960 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.994+0000 7fbde4ff9640 1 -- 172.21.15.31:0/3341248158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc00fae0 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.994+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde815f5f0 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.994+0000 7fbde4ff9640 1 -- 172.21.15.31:0/3341248158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00fe90 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.994+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde815fbb0 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.995+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8074900 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.995+0000 7fbde4ff9640 1 -- 172.21.15.31:0/3341248158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fbddc01d460 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.996+0000 7fbde4ff9640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbdd406af00 0x7fbdd406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.996+0000 7fbde4ff9640 1 -- 172.21.15.31:0/3341248158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbddc0897f0 con 0x7fbde80b5f10 2024-01-19T04:55:57.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.996+0000 7fbde7fff640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbdd406af00 0x7fbdd406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:57.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:57.996+0000 7fbde7fff640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbdd406af00 0x7fbdd406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8036000 tx=0x7fbdd8031000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:58.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.000+0000 7fbde4ff9640 1 -- 172.21.15.31:0/3341248158 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbddc054630 con 0x7fbde80b5f10 2024-01-19T04:55:58.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.177+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbde80667f0 con 0x7fbde80b5f10 2024-01-19T04:55:58.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.177+0000 7fbde4ff9640 1 -- 172.21.15.31:0/3341248158 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbddc057c50 con 0x7fbde80b5f10 2024-01-19T04:55:58.178 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbdd406af00 msgr2=0x7fbdd406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbdd406af00 0x7fbdd406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8036000 tx=0x7fbdd8031000 comp rx=0 tx=0).stop 2024-01-19T04:55:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbde80b5f10 msgr2=0x7fbde8151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbde80b5f10 0x7fbde8151730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0072c0 tx=0x7fbddc007780 comp rx=0 tx=0).stop 2024-01-19T04:55:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 shutdown_connections 2024-01-19T04:55:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbdd406af00 0x7fbdd406d3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbde81603b0 0x7fbde8160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbde80b5f10 0x7fbde8151730 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 --2- 172.21.15.31:0/3341248158 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbdd00039a0 0x7fbde80b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 >> 172.21.15.31:0/3341248158 conn(0x7fbde809c600 msgr2=0x7fbde80cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 shutdown_connections 2024-01-19T04:55:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.179+0000 7fbdee91d640 1 -- 172.21.15.31:0/3341248158 wait complete. 2024-01-19T04:55:58.190 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:55:58.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.292+0000 7f9040705640 1 Processor -- start 2024-01-19T04:55:58.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.292+0000 7f9040705640 1 -- start start 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f9040705640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f90380cf6f0 0x7f90380add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f9040705640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cfad0 0x7f90380cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f9040705640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f90380ccc70 0x7f90380b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f9040705640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90380b0a70 con 0x7f90380ccc70 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f9040705640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90380b18d0 con 0x7f90380cf6f0 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f9040705640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90380b0cc0 con 0x7f90380cfad0 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f903eca2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f90380ccc70 0x7f90380b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f903eca2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f90380ccc70 0x7f90380b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48604/0 (socket says 172.21.15.31:48604) 2024-01-19T04:55:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.293+0000 7f903eca2640 1 -- 172.21.15.31:0/1915400686 learned_addr learned my addr 172.21.15.31:0/1915400686 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903dca0640 1 --2- 172.21.15.31:0/1915400686 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cfad0 0x7f90380cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903e4a1640 1 --2- 172.21.15.31:0/1915400686 >> v2:172.21.15.137:3300/0 conn(0x7f90380cf6f0 0x7f90380add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903eca2640 1 -- 172.21.15.31:0/1915400686 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cfad0 msgr2=0x7f90380cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903eca2640 1 --2- 172.21.15.31:0/1915400686 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cfad0 0x7f90380cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903eca2640 1 -- 172.21.15.31:0/1915400686 >> v2:172.21.15.137:3300/0 conn(0x7f90380cf6f0 msgr2=0x7f90380add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903eca2640 1 --2- 172.21.15.31:0/1915400686 >> v2:172.21.15.137:3300/0 conn(0x7f90380cf6f0 0x7f90380add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903eca2640 1 -- 172.21.15.31:0/1915400686 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90380b0fa0 con 0x7f90380ccc70 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.294+0000 7f903eca2640 1 --2- 172.21.15.31:0/1915400686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380ccc70 0x7f90380b1d60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f903806b0c0 tx=0x7f903002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d42fd323d08d19 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f903d49f640 1 -- 172.21.15.31:0/1915400686 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903002ca70 con 0x7f90380ccc70 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f903d49f640 1 -- 172.21.15.31:0/1915400686 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903002cbd0 con 0x7f90380ccc70 2024-01-19T04:55:58.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f903d49f640 1 -- 172.21.15.31:0/1915400686 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90300343f0 con 0x7f90380ccc70 2024-01-19T04:55:58.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 -- 172.21.15.31:0/1915400686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380ccc70 msgr2=0x7f90380b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 --2- 172.21.15.31:0/1915400686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380ccc70 0x7f90380b1d60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f903806b0c0 tx=0x7f903002bed0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 -- 172.21.15.31:0/1915400686 shutdown_connections 2024-01-19T04:55:58.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 --2- 172.21.15.31:0/1915400686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380ccc70 0x7f90380b1d60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 --2- 172.21.15.31:0/1915400686 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cfad0 0x7f90380cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 --2- 172.21.15.31:0/1915400686 >> v2:172.21.15.137:3300/0 conn(0x7f90380cf6f0 0x7f90380add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 -- 172.21.15.31:0/1915400686 >> 172.21.15.31:0/1915400686 conn(0x7f903808afe0 msgr2=0x7f90380d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 -- 172.21.15.31:0/1915400686 shutdown_connections 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.295+0000 7f9040705640 1 -- 172.21.15.31:0/1915400686 wait complete. 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.296+0000 7f9040705640 1 Processor -- start 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.296+0000 7f9040705640 1 -- start start 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f9040705640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90380ccc70 0x7f90380af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f9040705640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cf6f0 0x7f903806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f9040705640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380cfad0 0x7f903806bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f9040705640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903815e850 con 0x7f90380cfad0 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f9040705640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903815d7b0 con 0x7f90380ccc70 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f9040705640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903815da90 con 0x7f90380cf6f0 2024-01-19T04:55:58.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f903eca2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380cfad0 0x7f903806bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f903eca2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380cfad0 0x7f903806bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48610/0 (socket says 172.21.15.31:48610) 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f903eca2640 1 -- 172.21.15.31:0/2503878161 learned_addr learned my addr 172.21.15.31:0/2503878161 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f903dca0640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cf6f0 0x7f903806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.297+0000 7f903e4a1640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90380ccc70 0x7f90380af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903eca2640 1 -- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cf6f0 msgr2=0x7f903806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903eca2640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cf6f0 0x7f903806b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903eca2640 1 -- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90380ccc70 msgr2=0x7f90380af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903eca2640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90380ccc70 0x7f90380af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903eca2640 1 -- 172.21.15.31:0/2503878161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9030009b30 con 0x7f90380cfad0 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903dca0640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cf6f0 0x7f903806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903e4a1640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90380ccc70 0x7f90380af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f903eca2640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380cfad0 0x7f903806bc00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9030036040 tx=0x7f9030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f902effd640 1 -- 172.21.15.31:0/2503878161 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030009510 con 0x7f90380cfad0 2024-01-19T04:55:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f902effd640 1 -- 172.21.15.31:0/2503878161 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903003e450 con 0x7f90380cfad0 2024-01-19T04:55:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f902effd640 1 -- 172.21.15.31:0/2503878161 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903003e800 con 0x7f90380cfad0 2024-01-19T04:55:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903815dd70 con 0x7f90380cfad0 2024-01-19T04:55:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.298+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903815e330 con 0x7f90380cfad0 2024-01-19T04:55:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.299+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903815e570 con 0x7f90380cfad0 2024-01-19T04:55:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.299+0000 7f902effd640 1 -- 172.21.15.31:0/2503878161 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f903003e960 con 0x7f90380cfad0 2024-01-19T04:55:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.300+0000 7f902effd640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f901c06af00 0x7f901c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.300+0000 7f902effd640 1 -- 172.21.15.31:0/2503878161 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90300ac3f0 con 0x7f90380cfad0 2024-01-19T04:55:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.300+0000 7f903e4a1640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f901c06af00 0x7f901c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.301+0000 7f903e4a1640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f901c06af00 0x7f901c06d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9034006fd0 tx=0x7f9034008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.308+0000 7f902effd640 1 -- 172.21.15.31:0/2503878161 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f903815e570 con 0x7f90380cfad0 2024-01-19T04:55:58.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.501+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f903815e570 con 0x7f90380cfad0 2024-01-19T04:55:58.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.501+0000 7f902effd640 1 -- 172.21.15.31:0/2503878161 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f903815e570 con 0x7f90380cfad0 2024-01-19T04:55:58.502 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:55:58.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.503+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f901c06af00 msgr2=0x7f901c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.503+0000 7f9040705640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f901c06af00 0x7f901c06d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9034006fd0 tx=0x7f9034008040 comp rx=0 tx=0).stop 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.503+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380cfad0 msgr2=0x7f903806bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.503+0000 7f9040705640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380cfad0 0x7f903806bc00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9030036040 tx=0x7f9030004a40 comp rx=0 tx=0).stop 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 shutdown_connections 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f901c06af00 0x7f901c06d3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90380cfad0 0x7f903806bc00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90380cf6f0 0x7f903806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 --2- 172.21.15.31:0/2503878161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90380ccc70 0x7f90380af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 >> 172.21.15.31:0/2503878161 conn(0x7f903808afe0 msgr2=0x7f90380add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 shutdown_connections 2024-01-19T04:55:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.504+0000 7f9040705640 1 -- 172.21.15.31:0/2503878161 wait complete. 2024-01-19T04:55:58.514 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi137.front.sepia.ceph.com... 2024-01-19T04:55:58.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:58.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-19T04:55:58.547 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:55:58.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:55:58.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:55:58.548 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-19T04:55:58.666 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:55:58.667 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:58.682 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:55:58.682 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:55:58.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:55:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.881+0000 7f654654c640 1 Processor -- start 2024-01-19T04:55:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.881+0000 7f654654c640 1 -- start start 2024-01-19T04:55:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f654654c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 0x7f654006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f654654c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f654015f0c0 0x7f654015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f654654c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654006bf30 con 0x7f654015eb70 2024-01-19T04:55:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f654654c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654015fa10 con 0x7f654015f0c0 2024-01-19T04:55:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f654654c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65400cb9e0 con 0x7f654015d8a0 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f654015eb70 0x7f65400cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55062/0 (socket says 172.21.15.31:55062) 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653f7fe640 1 -- 172.21.15.31:0/1622379390 learned_addr learned my addr 172.21.15.31:0/1622379390 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653ffff640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 0x7f654006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f6544ae9640 1 --2- 172.21.15.31:0/1622379390 >> v2:172.21.15.137:3300/0 conn(0x7f654015f0c0 0x7f654015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224998017 0 0) 0x7f654006bf30 con 0x7f654015eb70 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f652c003520 con 0x7f654015eb70 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2167811692 0 0) 0x7f652c003520 con 0x7f654015eb70 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 msgr2=0x7f654006b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 0x7f654006b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 >> v2:172.21.15.137:3300/0 conn(0x7f654015f0c0 msgr2=0x7f654015f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 --2- 172.21.15.31:0/1622379390 >> v2:172.21.15.137:3300/0 conn(0x7f654015f0c0 0x7f654015f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65400cbcc0 con 0x7f654015eb70 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f6528001670 con 0x7f654015eb70 2024-01-19T04:55:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 >> v1:172.21.15.31:6789/0 conn(0x7f654015eb70 legacy=0x7f65400cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.882+0000 7f653effd640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f652c003970 0x7f652c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f653effd640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 0x7f652c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f653effd640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652c004d00 0x7f652c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654015fa10 con 0x7f652c004d00 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65400cb9e0 con 0x7f652c003970 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c00b980 con 0x7f654015d8a0 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f6544ae9640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652c004d00 0x7f652c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f653ffff640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f652c003970 0x7f652c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f653f7fe640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 0x7f652c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f6544ae9640 1 -- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 msgr2=0x7f652c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f6544ae9640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f654015d8a0 0x7f652c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f6544ae9640 1 -- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f652c003970 msgr2=0x7f652c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f6544ae9640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f652c003970 0x7f652c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.883+0000 7f6544ae9640 1 -- 172.21.15.31:0/1622379390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c003520 con 0x7f652c004d00 2024-01-19T04:55:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f6544ae9640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652c004d00 0x7f652c005180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6538009fc0 tx=0x7f652c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9713f1bde07a816f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653802cc60 con 0x7f652c004d00 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653800f040 con 0x7f652c004d00 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f653effd640 1 -- 172.21.15.31:0/1622379390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538034440 con 0x7f652c004d00 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f654654c640 1 -- 172.21.15.31:0/1622379390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652c004d00 msgr2=0x7f652c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f654654c640 1 --2- 172.21.15.31:0/1622379390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652c004d00 0x7f652c005180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6538009fc0 tx=0x7f652c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f6544ae9640 1 -- 172.21.15.31:0/1622379390 reap_dead start 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f654654c640 1 -- 172.21.15.31:0/1622379390 shutdown_connections 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f654654c640 1 -- 172.21.15.31:0/1622379390 >> 172.21.15.31:0/1622379390 conn(0x7f654006abe0 msgr2=0x7f6540066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f654654c640 1 -- 172.21.15.31:0/1622379390 shutdown_connections 2024-01-19T04:55:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.884+0000 7f654654c640 1 -- 172.21.15.31:0/1622379390 wait complete. 2024-01-19T04:55:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.885+0000 7f654654c640 1 Processor -- start 2024-01-19T04:55:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f654654c640 1 -- start start 2024-01-19T04:55:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f654654c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65400c7500 0x7f6540160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f654654c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65401608c0 0x7f654014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f654654c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654014b0e0 0x7f65400d1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f654654c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540160d40 con 0x7f65400c7500 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f654654c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65400cb0e0 con 0x7f654014b0e0 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f654654c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65400ca7b0 con 0x7f65401608c0 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f653ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65400c7500 0x7f6540160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f653ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65400c7500 0x7f6540160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48638/0 (socket says 172.21.15.31:48638) 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.886+0000 7f653ffff640 1 -- 172.21.15.31:0/2464190776 learned_addr learned my addr 172.21.15.31:0/2464190776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653f7fe640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65401608c0 0x7f654014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f6544ae9640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654014b0e0 0x7f65400d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653ffff640 1 -- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65401608c0 msgr2=0x7f654014aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653ffff640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65401608c0 0x7f654014aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653ffff640 1 -- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654014b0e0 msgr2=0x7f65400d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653ffff640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654014b0e0 0x7f65400d1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653ffff640 1 -- 172.21.15.31:0/2464190776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65400caa90 con 0x7f65400c7500 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653ffff640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65400c7500 0x7f6540160380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f65300118e0 tx=0x7f6530011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653cff9640 1 -- 172.21.15.31:0/2464190776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653000faf0 con 0x7f65400c7500 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653cff9640 1 -- 172.21.15.31:0/2464190776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653000fc50 con 0x7f65400c7500 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65400cad70 con 0x7f65400c7500 2024-01-19T04:55:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f653cff9640 1 -- 172.21.15.31:0/2464190776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530014440 con 0x7f65400c7500 2024-01-19T04:55:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.887+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654014b560 con 0x7f65400c7500 2024-01-19T04:55:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.888+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65400696e0 con 0x7f65400c7500 2024-01-19T04:55:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.888+0000 7f653cff9640 1 -- 172.21.15.31:0/2464190776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f653001d400 con 0x7f65400c7500 2024-01-19T04:55:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.889+0000 7f653cff9640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f652006af00 0x7f652006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.889+0000 7f653cff9640 1 -- 172.21.15.31:0/2464190776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6530089f10 con 0x7f65400c7500 2024-01-19T04:55:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.889+0000 7f653f7fe640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f652006af00 0x7f652006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.890+0000 7f653f7fe640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f652006af00 0x7f652006d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6528004840 tx=0x7f6528009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:58.893+0000 7f653cff9640 1 -- 172.21.15.31:0/2464190776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6530054de0 con 0x7f65400c7500 2024-01-19T04:55:59.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.071+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65400b71e0 con 0x7f65400c7500 2024-01-19T04:55:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.072+0000 7f653cff9640 1 -- 172.21.15.31:0/2464190776 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f6530058400 con 0x7f65400c7500 2024-01-19T04:55:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:55:59.073 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:55:20.487566+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:55:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f652006af00 msgr2=0x7f652006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f652006af00 0x7f652006d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6528004840 tx=0x7f6528009040 comp rx=0 tx=0).stop 2024-01-19T04:55:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65400c7500 msgr2=0x7f6540160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65400c7500 0x7f6540160380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f65300118e0 tx=0x7f6530011da0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 shutdown_connections 2024-01-19T04:55:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f652006af00 0x7f652006d3c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654014b0e0 0x7f65400d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65401608c0 0x7f654014aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 --2- 172.21.15.31:0/2464190776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65400c7500 0x7f6540160380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 >> 172.21.15.31:0/2464190776 conn(0x7f654006abe0 msgr2=0x7f654015de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 shutdown_connections 2024-01-19T04:55:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.074+0000 7f654654c640 1 -- 172.21.15.31:0/2464190776 wait complete. 2024-01-19T04:55:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:55:59.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:55:59.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.189+0000 7f8c6755a640 1 Processor -- start 2024-01-19T04:55:59.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.189+0000 7f8c6755a640 1 -- start start 2024-01-19T04:55:59.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.190+0000 7f8c6755a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 0x7f8c600d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.190+0000 7f8c6755a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c600cf160 con 0x7f8c6015d720 2024-01-19T04:55:59.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.190+0000 7f8c6755a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c600ce8c0 con 0x7f8c600d3ea0 2024-01-19T04:55:59.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.190+0000 7f8c6755a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600ceba0 con 0x7f8c600d1c20 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.190+0000 7f8c652f6640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8c6015d720 0x7f8c600d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55074/0 (socket says 172.21.15.31:55074) 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.190+0000 7f8c652f6640 1 -- 172.21.15.31:0/1164562449 learned_addr learned my addr 172.21.15.31:0/1164562449 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c64af5640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 0x7f8c600d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556033887 0 0) 0x7f8c600cf160 con 0x7f8c6015d720 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c48003520 con 0x7f8c6015d720 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1346738887 0 0) 0x7f8c48003520 con 0x7f8c6015d720 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 msgr2=0x7f8c600d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 0x7f8c600d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 >> v1:172.21.15.137:6789/0 conn(0x7f8c600d3ea0 legacy=0x7f8c600b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c600cee80 con 0x7f8c6015d720 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f8c500019f0 con 0x7f8c6015d720 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 >> v1:172.21.15.31:6789/0 conn(0x7f8c6015d720 legacy=0x7f8c600d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c480039a0 0x7f8c48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 0x7f8c480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48004d00 0x7f8c48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600ceba0 con 0x7f8c480039a0 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4800bb40 con 0x7f8c48004d00 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4800bd00 con 0x7f8c600d1c20 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c652f6640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c480039a0 0x7f8c48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c65af7640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48004d00 0x7f8c48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c64af5640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 0x7f8c480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c652f6640 1 -- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 msgr2=0x7f8c480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c652f6640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600d1c20 0x7f8c480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c652f6640 1 -- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48004d00 msgr2=0x7f8c48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c652f6640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48004d00 0x7f8c48005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c652f6640 1 -- 172.21.15.31:0/1164562449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48003520 con 0x7f8c480039a0 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.191+0000 7f8c65af7640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48004d00 0x7f8c48005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.192+0000 7f8c652f6640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c480039a0 0x7f8c48003dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000a820 tx=0x7f8c480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a6f0802d56c5c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.192+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c500072d0 con 0x7f8c480039a0 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.192+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5002f070 con 0x7f8c480039a0 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.192+0000 7f8c5ffff640 1 -- 172.21.15.31:0/1164562449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50007690 con 0x7f8c480039a0 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.193+0000 7f8c6755a640 1 -- 172.21.15.31:0/1164562449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c480039a0 msgr2=0x7f8c48003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.193+0000 7f8c6755a640 1 --2- 172.21.15.31:0/1164562449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c480039a0 0x7f8c48003dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000a820 tx=0x7f8c480057f0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.193+0000 7f8c65af7640 1 -- 172.21.15.31:0/1164562449 reap_dead start 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.193+0000 7f8c6755a640 1 -- 172.21.15.31:0/1164562449 shutdown_connections 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.193+0000 7f8c6755a640 1 -- 172.21.15.31:0/1164562449 >> 172.21.15.31:0/1164562449 conn(0x7f8c600d3310 msgr2=0x7f8c600d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.193+0000 7f8c6755a640 1 -- 172.21.15.31:0/1164562449 shutdown_connections 2024-01-19T04:55:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.193+0000 7f8c6755a640 1 -- 172.21.15.31:0/1164562449 wait complete. 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.194+0000 7f8c6755a640 1 Processor -- start 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.194+0000 7f8c6755a640 1 -- start start 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c6755a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c480039a0 0x7f8c600b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c6755a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600b18b0 0x7f8c601f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c6755a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c601f6b80 0x7f8c600b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c6755a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c601f3e70 con 0x7f8c601f6b80 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c6755a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c601f35d0 con 0x7f8c480039a0 2024-01-19T04:55:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c6755a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c601f38b0 con 0x7f8c600b18b0 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c601f6b80 0x7f8c600b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c601f6b80 0x7f8c600b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48662/0 (socket says 172.21.15.31:48662) 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 -- 172.21.15.31:0/3049443617 learned_addr learned my addr 172.21.15.31:0/3049443617 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c652f6640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c480039a0 0x7f8c600b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c64af5640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600b18b0 0x7f8c601f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 -- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600b18b0 msgr2=0x7f8c601f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600b18b0 0x7f8c601f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 -- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c480039a0 msgr2=0x7f8c600b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c480039a0 0x7f8c600b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 -- 172.21.15.31:0/3049443617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c601f3b90 con 0x7f8c601f6b80 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c65af7640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c601f6b80 0x7f8c600b01f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8c6008d5a0 tx=0x7f8c5800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.195+0000 7f8c5dffb640 1 -- 172.21.15.31:0/3049443617 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c580052e0 con 0x7f8c601f6b80 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.196+0000 7f8c5dffb640 1 -- 172.21.15.31:0/3049443617 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58005460 con 0x7f8c601f6b80 2024-01-19T04:55:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.196+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c601f6660 con 0x7f8c601f6b80 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.196+0000 7f8c5dffb640 1 -- 172.21.15.31:0/3049443617 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58003830 con 0x7f8c601f6b80 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.196+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c601f7030 con 0x7f8c601f6b80 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.197+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c601f64c0 con 0x7f8c601f6b80 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.197+0000 7f8c5dffb640 1 -- 172.21.15.31:0/3049443617 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8c58003990 con 0x7f8c601f6b80 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.197+0000 7f8c5dffb640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c4c06aed0 0x7f8c4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.198+0000 7f8c5dffb640 1 -- 172.21.15.31:0/3049443617 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c58088ba0 con 0x7f8c601f6b80 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.198+0000 7f8c652f6640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c4c06aed0 0x7f8c4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.198+0000 7f8c652f6640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c4c06aed0 0x7f8c4c06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c50034000 tx=0x7f8c50036040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.202+0000 7f8c5dffb640 1 -- 172.21.15.31:0/3049443617 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8c601f64c0 con 0x7f8c601f6b80 2024-01-19T04:55:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.377+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c601f64c0 con 0x7f8c601f6b80 2024-01-19T04:55:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.378+0000 7f8c5dffb640 1 -- 172.21.15.31:0/3049443617 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c601f64c0 con 0x7f8c601f6b80 2024-01-19T04:55:59.378 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:55:59.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c4c06aed0 msgr2=0x7f8c4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c4c06aed0 0x7f8c4c06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c50034000 tx=0x7f8c50036040 comp rx=0 tx=0).stop 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c601f6b80 msgr2=0x7f8c600b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c601f6b80 0x7f8c600b01f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8c6008d5a0 tx=0x7f8c5800e820 comp rx=0 tx=0).stop 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 shutdown_connections 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8c4c06aed0 0x7f8c4c06d390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c601f6b80 0x7f8c600b01f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8c600b18b0 0x7f8c601f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 --2- 172.21.15.31:0/3049443617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c480039a0 0x7f8c600b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 >> 172.21.15.31:0/3049443617 conn(0x7f8c600d3310 msgr2=0x7f8c600cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:59.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 shutdown_connections 2024-01-19T04:55:59.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.380+0000 7f8c6755a640 1 -- 172.21.15.31:0/3049443617 wait complete. 2024-01-19T04:55:59.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:55:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.494+0000 7fc52f6d3640 1 Processor -- start 2024-01-19T04:55:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.494+0000 7fc52f6d3640 1 -- start start 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52f6d3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280cb2e0 0x7fc52815e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52f6d3640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc5280716f0 0x7fc52815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52f6d3640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52815ece0 con 0x7fc52815b510 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52f6d3640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc528071c20 con 0x7fc5280716f0 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52f6d3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc528071f00 con 0x7fc5280cb2e0 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52cc6e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc5280716f0 0x7fc52815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52d46f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280cb2e0 0x7fc52815e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52cc6e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc5280716f0 0x7fc52815afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:50076/0 (socket says 172.21.15.31:50076) 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52d46f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280cb2e0 0x7fc52815e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36248/0 (socket says 172.21.15.31:36248) 2024-01-19T04:55:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52cc6e640 1 -- 172.21.15.31:0/1991246965 learned_addr learned my addr 172.21.15.31:0/1991246965 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52dc70640 1 --1- 172.21.15.31:0/1991246965 >> v1:172.21.15.137:6789/0 conn(0x7fc52815b510 0x7fc52806d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:59856/0 (socket says 172.21.15.31:59856) 2024-01-19T04:55:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52cc6e640 1 -- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280cb2e0 msgr2=0x7fc52815e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52cc6e640 1 --2- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280cb2e0 0x7fc52815e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52cc6e640 1 -- 172.21.15.31:0/1991246965 >> v1:172.21.15.137:6789/0 conn(0x7fc52815b510 legacy=0x7fc52806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52cc6e640 1 -- 172.21.15.31:0/1991246965 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280721e0 con 0x7fc5280716f0 2024-01-19T04:55:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.495+0000 7fc52d46f640 1 --2- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280cb2e0 0x7fc52815e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:55:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.496+0000 7fc52cc6e640 1 --2- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280716f0 0x7fc52815afd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc518009870 tx=0x7fc51802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e34b5e9d4550cee server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.496+0000 7fc51ffff640 1 -- 172.21.15.31:0/1991246965 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518030840 con 0x7fc5280716f0 2024-01-19T04:55:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.496+0000 7fc51ffff640 1 -- 172.21.15.31:0/1991246965 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5180309a0 con 0x7fc5280716f0 2024-01-19T04:55:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.496+0000 7fc51ffff640 1 -- 172.21.15.31:0/1991246965 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518030d30 con 0x7fc5280716f0 2024-01-19T04:55:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 -- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280716f0 msgr2=0x7fc52815afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280716f0 0x7fc52815afd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc518009870 tx=0x7fc51802b980 comp rx=0 tx=0).stop 2024-01-19T04:55:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 -- 172.21.15.31:0/1991246965 shutdown_connections 2024-01-19T04:55:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280716f0 0x7fc52815afd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/1991246965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280cb2e0 0x7fc52815e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 -- 172.21.15.31:0/1991246965 >> 172.21.15.31:0/1991246965 conn(0x7fc52806aa00 msgr2=0x7fc52806ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 -- 172.21.15.31:0/1991246965 shutdown_connections 2024-01-19T04:55:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.497+0000 7fc52f6d3640 1 -- 172.21.15.31:0/1991246965 wait complete. 2024-01-19T04:55:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.498+0000 7fc52f6d3640 1 Processor -- start 2024-01-19T04:55:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.498+0000 7fc52f6d3640 1 -- start start 2024-01-19T04:55:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52f6d3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280716f0 0x7fc52813bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52f6d3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280cb2e0 0x7fc5281421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52f6d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc52815b510 0x7fc5281426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52f6d3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52806c4a0 con 0x7fc52815b510 2024-01-19T04:55:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52f6d3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528140cd0 con 0x7fc5280cb2e0 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52f6d3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52806c6f0 con 0x7fc5280716f0 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52dc70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc52815b510 0x7fc5281426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52dc70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc52815b510 0x7fc5281426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39704/0 (socket says 172.21.15.31:39704) 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52dc70640 1 -- 172.21.15.31:0/5699057 learned_addr learned my addr 172.21.15.31:0/5699057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52d46f640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280716f0 0x7fc52813bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.499+0000 7fc52cc6e640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280cb2e0 0x7fc5281421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52dc70640 1 -- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280716f0 msgr2=0x7fc52813bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52dc70640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280716f0 0x7fc52813bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52dc70640 1 -- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280cb2e0 msgr2=0x7fc5281421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52dc70640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280cb2e0 0x7fc5281421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52dc70640 1 -- 172.21.15.31:0/5699057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524009400 con 0x7fc52815b510 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52dc70640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc52815b510 0x7fc5281426e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c6f0 tx=0x7fc52000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc51dffb640 1 -- 172.21.15.31:0/5699057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520005da0 con 0x7fc52815b510 2024-01-19T04:55:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc51dffb640 1 -- 172.21.15.31:0/5699057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52000a640 con 0x7fc52815b510 2024-01-19T04:55:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5180094d0 con 0x7fc52815b510 2024-01-19T04:55:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc51dffb640 1 -- 172.21.15.31:0/5699057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52000a9b0 con 0x7fc52815b510 2024-01-19T04:55:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.500+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52806cf90 con 0x7fc52815b510 2024-01-19T04:55:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.501+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5281409f0 con 0x7fc52815b510 2024-01-19T04:55:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.502+0000 7fc51dffb640 1 -- 172.21.15.31:0/5699057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc52000ab10 con 0x7fc52815b510 2024-01-19T04:55:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.502+0000 7fc51dffb640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc50c06aed0 0x7fc50c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.503+0000 7fc51dffb640 1 -- 172.21.15.31:0/5699057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc520088bd0 con 0x7fc52815b510 2024-01-19T04:55:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.503+0000 7fc52d46f640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc50c06aed0 0x7fc50c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.503+0000 7fc52d46f640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc50c06aed0 0x7fc50c06d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc524009cd0 tx=0x7fc524009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.507+0000 7fc51dffb640 1 -- 172.21.15.31:0/5699057 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc520053be0 con 0x7fc52815b510 2024-01-19T04:55:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.685+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc528074900 con 0x7fc52815b510 2024-01-19T04:55:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.686+0000 7fc51dffb640 1 -- 172.21.15.31:0/5699057 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc5200170a0 con 0x7fc52815b510 2024-01-19T04:55:59.686 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:55:59.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc50c06aed0 msgr2=0x7fc50c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc50c06aed0 0x7fc50c06d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc524009cd0 tx=0x7fc524009190 comp rx=0 tx=0).stop 2024-01-19T04:55:59.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc52815b510 msgr2=0x7fc5281426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc52815b510 0x7fc5281426e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c6f0 tx=0x7fc52000cbb0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 shutdown_connections 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc50c06aed0 0x7fc50c06d390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc52815b510 0x7fc5281426e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5280cb2e0 0x7fc5281421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 --2- 172.21.15.31:0/5699057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5280716f0 0x7fc52813bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.687+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 >> 172.21.15.31:0/5699057 conn(0x7fc52806aa00 msgr2=0x7fc52806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.688+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 shutdown_connections 2024-01-19T04:55:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.688+0000 7fc52f6d3640 1 -- 172.21.15.31:0/5699057 wait complete. 2024-01-19T04:55:59.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:55:59.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.801+0000 7efcceb6f640 1 Processor -- start 2024-01-19T04:55:59.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.801+0000 7efcceb6f640 1 -- start start 2024-01-19T04:55:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efcceb6f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efcc80add00 0x7efcc80cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efcceb6f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efcc80ccc70 0x7efcc80b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efcceb6f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc80b0a70 con 0x7efcc80cf6f0 2024-01-19T04:55:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efcceb6f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc80cd080 con 0x7efcc80ccc70 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efcceb6f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc80b18d0 con 0x7efcc80add00 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efccd10c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efcc80ccc70 0x7efcc80b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efccd10c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efcc80ccc70 0x7efcc80b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39710/0 (socket says 172.21.15.31:39710) 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.802+0000 7efccd10c640 1 -- 172.21.15.31:0/282572083 learned_addr learned my addr 172.21.15.31:0/282572083 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.803+0000 7efcc7fff640 1 --2- 172.21.15.31:0/282572083 >> v2:172.21.15.137:3300/0 conn(0x7efcc80add00 0x7efcc80cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.803+0000 7efccd10c640 1 -- 172.21.15.31:0/282572083 >> v2:172.21.15.137:3300/0 conn(0x7efcc80add00 msgr2=0x7efcc80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.803+0000 7efccd10c640 1 --2- 172.21.15.31:0/282572083 >> v2:172.21.15.137:3300/0 conn(0x7efcc80add00 0x7efcc80cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.803+0000 7efccd10c640 1 -- 172.21.15.31:0/282572083 >> v1:172.21.15.137:6789/0 conn(0x7efcc80cf6f0 legacy=0x7efcc80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.803+0000 7efccd10c640 1 -- 172.21.15.31:0/282572083 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc80ae160 con 0x7efcc80ccc70 2024-01-19T04:55:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.803+0000 7efcc7fff640 1 --2- 172.21.15.31:0/282572083 >> v2:172.21.15.137:3300/0 conn(0x7efcc80add00 0x7efcc80cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.803+0000 7efccd10c640 1 --2- 172.21.15.31:0/282572083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80ccc70 0x7efcc80b1d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efcc815efc0 tx=0x7efcb802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70af883d6576961c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcc77fe640 1 -- 172.21.15.31:0/282572083 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8030860 con 0x7efcc80ccc70 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcc77fe640 1 -- 172.21.15.31:0/282572083 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb80309c0 con 0x7efcc80ccc70 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcc77fe640 1 -- 172.21.15.31:0/282572083 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8030d50 con 0x7efcc80ccc70 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 -- 172.21.15.31:0/282572083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80ccc70 msgr2=0x7efcc80b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 --2- 172.21.15.31:0/282572083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80ccc70 0x7efcc80b1d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efcc815efc0 tx=0x7efcb802b9a0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 -- 172.21.15.31:0/282572083 shutdown_connections 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 --2- 172.21.15.31:0/282572083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80ccc70 0x7efcc80b1d60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 --2- 172.21.15.31:0/282572083 >> v2:172.21.15.137:3300/0 conn(0x7efcc80add00 0x7efcc80cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 -- 172.21.15.31:0/282572083 >> 172.21.15.31:0/282572083 conn(0x7efcc808afe0 msgr2=0x7efcc80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 -- 172.21.15.31:0/282572083 shutdown_connections 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.804+0000 7efcceb6f640 1 -- 172.21.15.31:0/282572083 wait complete. 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.805+0000 7efcceb6f640 1 Processor -- start 2024-01-19T04:55:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.805+0000 7efcceb6f640 1 -- start start 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efcceb6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80add00 0x7efcc80b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efcceb6f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc80ccc70 0x7efcc814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efcceb6f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcc80cf6f0 0x7efcc80b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efcceb6f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80b5be0 con 0x7efcc80add00 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efcceb6f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc814caa0 con 0x7efcc80ccc70 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efcceb6f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80b4b40 con 0x7efcc80cf6f0 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efccc90b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80add00 0x7efcc80b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efccc90b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80add00 0x7efcc80b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39726/0 (socket says 172.21.15.31:39726) 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.806+0000 7efccc90b640 1 -- 172.21.15.31:0/4218039530 learned_addr learned my addr 172.21.15.31:0/4218039530 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efccd10c640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcc80cf6f0 0x7efcc80b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efcc7fff640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc80ccc70 0x7efcc814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efccc90b640 1 -- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcc80cf6f0 msgr2=0x7efcc80b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efccc90b640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcc80cf6f0 0x7efcc80b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efccc90b640 1 -- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc80ccc70 msgr2=0x7efcc814c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efccc90b640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc80ccc70 0x7efcc814c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efccc90b640 1 -- 172.21.15.31:0/4218039530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80094d0 con 0x7efcc80add00 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efcc7fff640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc80ccc70 0x7efcc814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efccc90b640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80add00 0x7efcc80b6000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c760 tx=0x7efcc000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efcc57fa640 1 -- 172.21.15.31:0/4218039530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0006d50 con 0x7efcc80add00 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efcc57fa640 1 -- 172.21.15.31:0/4218039530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc000ce70 con 0x7efcc80add00 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc80b4860 con 0x7efcc80add00 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efcc57fa640 1 -- 172.21.15.31:0/4218039530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0003b90 con 0x7efcc80add00 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.807+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc80b5800 con 0x7efcc80add00 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.808+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc80b5d20 con 0x7efcc80add00 2024-01-19T04:55:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.808+0000 7efcc57fa640 1 -- 172.21.15.31:0/4218039530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efcc0003cf0 con 0x7efcc80add00 2024-01-19T04:55:59.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.809+0000 7efcc57fa640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efcb406aed0 0x7efcb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:55:59.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.809+0000 7efcc57fa640 1 -- 172.21.15.31:0/4218039530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcc0089170 con 0x7efcc80add00 2024-01-19T04:55:59.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.809+0000 7efcc7fff640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efcb406aed0 0x7efcb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:55:59.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.810+0000 7efcc7fff640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efcb406aed0 0x7efcb406d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efcc815ebf0 tx=0x7efcbc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:55:59.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.813+0000 7efcc57fa640 1 -- 172.21.15.31:0/4218039530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efcc80b5d20 con 0x7efcc80add00 2024-01-19T04:55:59.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.991+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcc80b5d20 con 0x7efcc80add00 2024-01-19T04:55:59.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.991+0000 7efcc57fa640 1 -- 172.21.15.31:0/4218039530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7efcc80b5d20 con 0x7efcc80add00 2024-01-19T04:55:59.992 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:55:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efcb406aed0 msgr2=0x7efcb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efcb406aed0 0x7efcb406d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efcc815ebf0 tx=0x7efcbc008040 comp rx=0 tx=0).stop 2024-01-19T04:55:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80add00 msgr2=0x7efcc80b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:55:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80add00 0x7efcc80b6000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c760 tx=0x7efcc000cc20 comp rx=0 tx=0).stop 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 shutdown_connections 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efcb406aed0 0x7efcb406d390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcc80cf6f0 0x7efcc80b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc80ccc70 0x7efcc814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 --2- 172.21.15.31:0/4218039530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcc80add00 0x7efcc80b6000 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 >> 172.21.15.31:0/4218039530 conn(0x7efcc808afe0 msgr2=0x7efcc80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 shutdown_connections 2024-01-19T04:55:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:55:59.994+0000 7efcceb6f640 1 -- 172.21.15.31:0/4218039530 wait complete. 2024-01-19T04:56:00.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:00.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.106+0000 7f4097043640 1 Processor -- start 2024-01-19T04:56:00.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.107+0000 7f4097043640 1 -- start start 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.107+0000 7f4097043640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f40900d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.107+0000 7f4097043640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f40900d1c20 0x7f40900b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.107+0000 7f4097043640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900cf160 con 0x7f40900d3ea0 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.107+0000 7f4097043640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40900ce8c0 con 0x7f40900d1c20 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.107+0000 7f4097043640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40900ceba0 con 0x7f409015d720 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f40955e0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f40900d1c20 0x7f40900b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f4094ddf640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f40900d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f408ffff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f40900d3ea0 0x7f40900d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:59874/0 (socket says 172.21.15.31:59874) 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f408ffff640 1 -- 172.21.15.31:0/907648807 learned_addr learned my addr 172.21.15.31:0/907648807 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:00.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f4094ddf640 1 -- 172.21.15.31:0/907648807 >> v2:172.21.15.137:3300/0 conn(0x7f40900d1c20 msgr2=0x7f40900b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:00.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f4094ddf640 1 --2- 172.21.15.31:0/907648807 >> v2:172.21.15.137:3300/0 conn(0x7f40900d1c20 0x7f40900b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f4094ddf640 1 -- 172.21.15.31:0/907648807 >> v1:172.21.15.137:6789/0 conn(0x7f40900d3ea0 legacy=0x7f40900d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f4094ddf640 1 -- 172.21.15.31:0/907648807 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900cee80 con 0x7f409015d720 2024-01-19T04:56:00.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.108+0000 7f40955e0640 1 --2- 172.21.15.31:0/907648807 >> v2:172.21.15.137:3300/0 conn(0x7f40900d1c20 0x7f40900b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.109+0000 7f4094ddf640 1 --2- 172.21.15.31:0/907648807 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f40900d0ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40840097a0 tx=0x7f408402b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5fa585315a9ccd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.109+0000 7f408f7fe640 1 -- 172.21.15.31:0/907648807 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408402f050 con 0x7f409015d720 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.109+0000 7f408f7fe640 1 -- 172.21.15.31:0/907648807 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084031700 con 0x7f409015d720 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.109+0000 7f408f7fe640 1 -- 172.21.15.31:0/907648807 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084031a90 con 0x7f409015d720 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 -- 172.21.15.31:0/907648807 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 msgr2=0x7f40900d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 --2- 172.21.15.31:0/907648807 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f40900d0ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40840097a0 tx=0x7f408402b7a0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 -- 172.21.15.31:0/907648807 shutdown_connections 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 --2- 172.21.15.31:0/907648807 >> v2:172.21.15.137:3300/0 conn(0x7f40900d1c20 0x7f40900b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 --2- 172.21.15.31:0/907648807 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f40900d0ee0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 -- 172.21.15.31:0/907648807 >> 172.21.15.31:0/907648807 conn(0x7f40900d3310 msgr2=0x7f40900d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 -- 172.21.15.31:0/907648807 shutdown_connections 2024-01-19T04:56:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.110+0000 7f4097043640 1 -- 172.21.15.31:0/907648807 wait complete. 2024-01-19T04:56:00.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.111+0000 7f4097043640 1 Processor -- start 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.111+0000 7f4097043640 1 -- start start 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4097043640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40900d1c20 0x7f4090150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4097043640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40900d3ea0 0x7f409015f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4097043640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f409015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4097043640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40901f2d20 con 0x7f40900d1c20 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4097043640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4090151240 con 0x7f40900d3ea0 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4097043640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409015fc40 con 0x7f409015d720 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40900d1c20 0x7f4090150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40900d1c20 0x7f4090150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39742/0 (socket says 172.21.15.31:39742) 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 -- 172.21.15.31:0/1497052316 learned_addr learned my addr 172.21.15.31:0/1497052316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f40955e0640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f409015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f408ffff640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40900d3ea0 0x7f409015f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 -- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 msgr2=0x7f409015f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f409015f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 -- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40900d3ea0 msgr2=0x7f409015f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40900d3ea0 0x7f409015f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 -- 172.21.15.31:0/1497052316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088009a70 con 0x7f40900d1c20 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.112+0000 7f4094ddf640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40900d1c20 0x7f4090150100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40840098c0 tx=0x7f40840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.113+0000 7f408d7fa640 1 -- 172.21.15.31:0/1497052316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084031500 con 0x7f40900d1c20 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.113+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4084009400 con 0x7f40900d1c20 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.113+0000 7f408d7fa640 1 -- 172.21.15.31:0/1497052316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40840339b0 con 0x7f40900d1c20 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.113+0000 7f408d7fa640 1 -- 172.21.15.31:0/1497052316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084033d60 con 0x7f40900d1c20 2024-01-19T04:56:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.113+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4090150bf0 con 0x7f40900d1c20 2024-01-19T04:56:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.114+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4090074900 con 0x7f40900d1c20 2024-01-19T04:56:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.114+0000 7f408d7fa640 1 -- 172.21.15.31:0/1497052316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f408403a040 con 0x7f40900d1c20 2024-01-19T04:56:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.114+0000 7f408d7fa640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f407406aed0 0x7f407406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.115+0000 7f408d7fa640 1 -- 172.21.15.31:0/1497052316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40840aa670 con 0x7f40900d1c20 2024-01-19T04:56:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.115+0000 7f408ffff640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f407406aed0 0x7f407406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.115+0000 7f408ffff640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f407406aed0 0x7f407406d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40900a1390 tx=0x7f407c007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.119+0000 7f408d7fa640 1 -- 172.21.15.31:0/1497052316 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4084075570 con 0x7f40900d1c20 2024-01-19T04:56:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.309+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f40900d02a0 con 0x7f40900d1c20 2024-01-19T04:56:00.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.309+0000 7f408d7fa640 1 -- 172.21.15.31:0/1497052316 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4084078b90 con 0x7f40900d1c20 2024-01-19T04:56:00.310 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f407406aed0 msgr2=0x7f407406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f407406aed0 0x7f407406d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40900a1390 tx=0x7f407c007e80 comp rx=0 tx=0).stop 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40900d1c20 msgr2=0x7f4090150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40900d1c20 0x7f4090150100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40840098c0 tx=0x7f40840046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 shutdown_connections 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f407406aed0 0x7f407406d390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f409015d720 0x7f409015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40900d3ea0 0x7f409015f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 --2- 172.21.15.31:0/1497052316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40900d1c20 0x7f4090150100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 >> 172.21.15.31:0/1497052316 conn(0x7f40900d3310 msgr2=0x7f40900cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 shutdown_connections 2024-01-19T04:56:00.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.311+0000 7f4097043640 1 -- 172.21.15.31:0/1497052316 wait complete. 2024-01-19T04:56:00.322 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:00.322 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:00.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:00.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.477+0000 7fe340b7d640 1 Processor -- start 2024-01-19T04:56:00.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.477+0000 7fe340b7d640 1 -- start start 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe340b7d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe33c0cfd10 0x7fe33c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe340b7d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe33c0d0ef0 0x7fe33c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe340b7d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33c0d06b0 con 0x7fe33c160bb0 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe340b7d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33c0d18c0 con 0x7fe33c0cfd10 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe340b7d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33c0d1b90 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe339d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe33c0cfd10 0x7fe33c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe33a59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe33c160bb0 0x7fe33c0b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58336/0 (socket says 172.21.15.31:58336) 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe33a59c640 1 -- 172.21.15.31:0/3132234524 learned_addr learned my addr 172.21.15.31:0/3132234524 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.478+0000 7fe339d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe33c0cfd10 0x7fe33c0d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39748/0 (socket says 172.21.15.31:39748) 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe33ad9d640 1 --2- 172.21.15.31:0/3132234524 >> v2:172.21.15.137:3300/0 conn(0x7fe33c0d0ef0 0x7fe33c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe339d9b640 1 -- 172.21.15.31:0/3132234524 >> v2:172.21.15.137:3300/0 conn(0x7fe33c0d0ef0 msgr2=0x7fe33c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe339d9b640 1 --2- 172.21.15.31:0/3132234524 >> v2:172.21.15.137:3300/0 conn(0x7fe33c0d0ef0 0x7fe33c0d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe339d9b640 1 -- 172.21.15.31:0/3132234524 >> v1:172.21.15.31:6789/0 conn(0x7fe33c160bb0 legacy=0x7fe33c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe339d9b640 1 -- 172.21.15.31:0/3132234524 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c0d1e70 con 0x7fe33c0cfd10 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe33959a640 1 -- 172.21.15.31:0/3132234524 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260498708 0 0) 0x7fe33c0d06b0 con 0x7fe33c160bb0 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe33ad9d640 1 --2- 172.21.15.31:0/3132234524 >> v2:172.21.15.137:3300/0 conn(0x7fe33c0d0ef0 0x7fe33c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe339d9b640 1 --2- 172.21.15.31:0/3132234524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0cfd10 0x7fe33c0d0170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0b93b0 tx=0x7fe32402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2982e708fc11117b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe33959a640 1 -- 172.21.15.31:0/3132234524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324031830 con 0x7fe33c0cfd10 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe33959a640 1 -- 172.21.15.31:0/3132234524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe324031990 con 0x7fe33c0cfd10 2024-01-19T04:56:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.479+0000 7fe33959a640 1 -- 172.21.15.31:0/3132234524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324031d20 con 0x7fe33c0cfd10 2024-01-19T04:56:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 -- 172.21.15.31:0/3132234524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0cfd10 msgr2=0x7fe33c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 --2- 172.21.15.31:0/3132234524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0cfd10 0x7fe33c0d0170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0b93b0 tx=0x7fe32402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 -- 172.21.15.31:0/3132234524 shutdown_connections 2024-01-19T04:56:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 --2- 172.21.15.31:0/3132234524 >> v2:172.21.15.137:3300/0 conn(0x7fe33c0d0ef0 0x7fe33c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 --2- 172.21.15.31:0/3132234524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0cfd10 0x7fe33c0d0170 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 -- 172.21.15.31:0/3132234524 >> 172.21.15.31:0/3132234524 conn(0x7fe33c09c600 msgr2=0x7fe33c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 -- 172.21.15.31:0/3132234524 shutdown_connections 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.480+0000 7fe340b7d640 1 -- 172.21.15.31:0/3132234524 wait complete. 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.481+0000 7fe340b7d640 1 Processor -- start 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.481+0000 7fe340b7d640 1 -- start start 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe340b7d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe33c0cfd10 0x7fe33c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe340b7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0d0ef0 0x7fe33c14a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe340b7d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe33c160bb0 0x7fe33c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe340b7d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c15e6c0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe340b7d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c15e910 con 0x7fe33c0cfd10 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe340b7d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c15ebf0 con 0x7fe33c160bb0 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe339d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0d0ef0 0x7fe33c14a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe339d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0d0ef0 0x7fe33c14a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39754/0 (socket says 172.21.15.31:39754) 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe339d9b640 1 -- 172.21.15.31:0/2975914111 learned_addr learned my addr 172.21.15.31:0/2975914111 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe339d9b640 1 -- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe33c160bb0 msgr2=0x7fe33c0b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.482+0000 7fe339d9b640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe33c160bb0 0x7fe33c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe339d9b640 1 -- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe33c0cfd10 msgr2=0x7fe33c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe339d9b640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe33c0cfd10 0x7fe33c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe339d9b640 1 -- 172.21.15.31:0/2975914111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324009400 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe339d9b640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0d0ef0 0x7fe33c14a290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe324009890 tx=0x7fe3240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe332ffd640 1 -- 172.21.15.31:0/2975914111 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32403ba70 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe332ffd640 1 -- 172.21.15.31:0/2975914111 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32403bbf0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c15eed0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe332ffd640 1 -- 172.21.15.31:0/2975914111 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240453f0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.483+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c0caf90 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.484+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c15f3f0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.484+0000 7fe332ffd640 1 -- 172.21.15.31:0/2975914111 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe324045550 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.485+0000 7fe332ffd640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe31c06aed0 0x7fe31c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.485+0000 7fe332ffd640 1 -- 172.21.15.31:0/2975914111 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3240aba40 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.485+0000 7fe33a59c640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe31c06aed0 0x7fe31c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.485+0000 7fe33a59c640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe31c06aed0 0x7fe31c06d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe334007fd0 tx=0x7fe334007960 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.489+0000 7fe332ffd640 1 -- 172.21.15.31:0/2975914111 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe33c15f3f0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.675+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe33c15f3f0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.676+0000 7fe332ffd640 1 -- 172.21.15.31:0/2975914111 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16720 (secure 0 0 0) 0x7fe33c15f3f0 con 0x7fe33c0d0ef0 2024-01-19T04:56:00.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:00.677 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:00.571872+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe31c06aed0 msgr2=0x7fe31c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe31c06aed0 0x7fe31c06d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe334007fd0 tx=0x7fe334007960 comp rx=0 tx=0).stop 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0d0ef0 msgr2=0x7fe33c14a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0d0ef0 0x7fe33c14a290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe324009890 tx=0x7fe3240046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 shutdown_connections 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe31c06aed0 0x7fe31c06d390 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe33c160bb0 0x7fe33c0b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe33c0d0ef0 0x7fe33c14a290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 --2- 172.21.15.31:0/2975914111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe33c0cfd10 0x7fe33c0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 >> 172.21.15.31:0/2975914111 conn(0x7fe33c09c600 msgr2=0x7fe33c06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 shutdown_connections 2024-01-19T04:56:00.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.678+0000 7fe340b7d640 1 -- 172.21.15.31:0/2975914111 wait complete. 2024-01-19T04:56:00.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:00.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:00.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.794+0000 7f5f57865640 1 Processor -- start 2024-01-19T04:56:00.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.794+0000 7f5f57865640 1 -- start start 2024-01-19T04:56:00.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.795+0000 7f5f57865640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5f5015d720 0x7f5f500d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.795+0000 7f5f57865640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f500cf160 con 0x7f5f500d1c20 2024-01-19T04:56:00.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.795+0000 7f5f57865640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f500ce8c0 con 0x7f5f500d3ea0 2024-01-19T04:56:00.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.795+0000 7f5f57865640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f500ceba0 con 0x7f5f5015d720 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.795+0000 7f5f55e02640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5f500d1c20 0x7f5f500b3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58340/0 (socket says 172.21.15.31:58340) 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.795+0000 7f5f55e02640 1 -- 172.21.15.31:0/403886742 learned_addr learned my addr 172.21.15.31:0/403886742 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f55601640 1 --2- 172.21.15.31:0/403886742 >> v2:172.21.15.137:3300/0 conn(0x7f5f5015d720 0x7f5f500d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200925070 0 0) 0x7f5f500cf160 con 0x7f5f500d1c20 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f38003520 con 0x7f5f500d1c20 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245526627 0 0) 0x7f5f500ce8c0 con 0x7f5f500d3ea0 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f500cf160 con 0x7f5f500d3ea0 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f55601640 1 -- 172.21.15.31:0/403886742 >> v1:172.21.15.137:6789/0 conn(0x7f5f500d3ea0 legacy=0x7f5f500d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 587576100 0 0) 0x7f5f38003520 con 0x7f5f500d1c20 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f55601640 1 -- 172.21.15.31:0/403886742 >> v1:172.21.15.31:6789/0 conn(0x7f5f500d1c20 legacy=0x7f5f500b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.796+0000 7f5f55601640 1 -- 172.21.15.31:0/403886742 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f500cee80 con 0x7f5f5015d720 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.797+0000 7f5f55601640 1 --2- 172.21.15.31:0/403886742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 0x7f5f500d0ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f500cf350 tx=0x7f5f4802b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601080523670b87c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.797+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f400019f0 con 0x7f5f5015d720 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.797+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4802e050 con 0x7f5f5015d720 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.797+0000 7f5f4ffff640 1 -- 172.21.15.31:0/403886742 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48030710 con 0x7f5f5015d720 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.798+0000 7f5f57865640 1 -- 172.21.15.31:0/403886742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 msgr2=0x7f5f500d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.798+0000 7f5f57865640 1 --2- 172.21.15.31:0/403886742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 0x7f5f500d0ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f500cf350 tx=0x7f5f4802b6f0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.798+0000 7f5f57865640 1 -- 172.21.15.31:0/403886742 shutdown_connections 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.798+0000 7f5f57865640 1 --2- 172.21.15.31:0/403886742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 0x7f5f500d0ee0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.798+0000 7f5f57865640 1 -- 172.21.15.31:0/403886742 >> 172.21.15.31:0/403886742 conn(0x7f5f500d3310 msgr2=0x7f5f500d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.798+0000 7f5f57865640 1 -- 172.21.15.31:0/403886742 shutdown_connections 2024-01-19T04:56:00.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.798+0000 7f5f57865640 1 -- 172.21.15.31:0/403886742 wait complete. 2024-01-19T04:56:00.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.799+0000 7f5f57865640 1 Processor -- start 2024-01-19T04:56:00.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.799+0000 7f5f57865640 1 -- start start 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f57865640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f500d1c20 0x7f5f50159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f57865640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f500d3ea0 0x7f5f5015f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f57865640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 0x7f5f500b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f57865640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f500b7340 con 0x7f5f500d1c20 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f57865640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5015fd40 con 0x7f5f5015d720 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f57865640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f500b6a50 con 0x7f5f500d3ea0 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f500d1c20 0x7f5f50159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f500d1c20 0x7f5f50159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39756/0 (socket says 172.21.15.31:39756) 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 -- 172.21.15.31:0/613279085 learned_addr learned my addr 172.21.15.31:0/613279085 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f54e00640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f500d3ea0 0x7f5f5015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 -- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f500d3ea0 msgr2=0x7f5f5015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55e02640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 0x7f5f500b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f500d3ea0 0x7f5f5015f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 -- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 msgr2=0x7f5f500b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 0x7f5f500b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 -- 172.21.15.31:0/613279085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48009400 con 0x7f5f500d1c20 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f55601640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f500d1c20 0x7f5f50159cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f4dffb640 1 -- 172.21.15.31:0/613279085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4803c880 con 0x7f5f500d1c20 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f4dffb640 1 -- 172.21.15.31:0/613279085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4803c9e0 con 0x7f5f500d1c20 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.800+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5015faa0 con 0x7f5f500d1c20 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.801+0000 7f5f4dffb640 1 -- 172.21.15.31:0/613279085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4803cd90 con 0x7f5f500d1c20 2024-01-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.801+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f500b6fd0 con 0x7f5f500d1c20 2024-01-19T04:56:00.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.802+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f50074900 con 0x7f5f500d1c20 2024-01-19T04:56:00.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.802+0000 7f5f4dffb640 1 -- 172.21.15.31:0/613279085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5f48039040 con 0x7f5f500d1c20 2024-01-19T04:56:00.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.803+0000 7f5f4dffb640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f3c06aed0 0x7f5f3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:00.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.803+0000 7f5f4dffb640 1 -- 172.21.15.31:0/613279085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5f480ab2a0 con 0x7f5f500d1c20 2024-01-19T04:56:00.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.803+0000 7f5f54e00640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f3c06aed0 0x7f5f3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:00.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.804+0000 7f5f54e00640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f3c06aed0 0x7f5f3c06d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5f440010e0 tx=0x7f5f44009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.807+0000 7f5f4dffb640 1 -- 172.21.15.31:0/613279085 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5f480760e0 con 0x7f5f500d1c20 2024-01-19T04:56:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.983+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f500c9c70 con 0x7f5f500d1c20 2024-01-19T04:56:00.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.983+0000 7f5f4dffb640 1 -- 172.21.15.31:0/613279085 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f48037020 con 0x7f5f500d1c20 2024-01-19T04:56:00.984 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f3c06aed0 msgr2=0x7f5f3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f3c06aed0 0x7f5f3c06d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5f440010e0 tx=0x7f5f44009040 comp rx=0 tx=0).stop 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f500d1c20 msgr2=0x7f5f50159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f500d1c20 0x7f5f50159cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f480046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 shutdown_connections 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f3c06aed0 0x7f5f3c06d390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f5015d720 0x7f5f500b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f500d3ea0 0x7f5f5015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 --2- 172.21.15.31:0/613279085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f500d1c20 0x7f5f50159cf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 >> 172.21.15.31:0/613279085 conn(0x7f5f500d3310 msgr2=0x7f5f500b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 shutdown_connections 2024-01-19T04:56:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:00.985+0000 7f5f57865640 1 -- 172.21.15.31:0/613279085 wait complete. 2024-01-19T04:56:00.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.100+0000 7f07ea81e640 1 Processor -- start 2024-01-19T04:56:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.100+0000 7f07ea81e640 1 -- start start 2024-01-19T04:56:01.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.100+0000 7f07ea81e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 0x7f07e415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.100+0000 7f07ea81e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e415dee0 con 0x7f07e40d0ef0 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.100+0000 7f07ea81e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e415e1c0 con 0x7f07e40d3e50 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.100+0000 7f07ea81e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e415e4a0 con 0x7f07e40d18e0 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e8dbb640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f07e40d0ef0 0x7f07e40cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58354/0 (socket says 172.21.15.31:58354) 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e8dbb640 1 -- 172.21.15.31:0/2873885844 learned_addr learned my addr 172.21.15.31:0/2873885844 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e3fff640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 0x7f07e415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080329860 0 0) 0x7f07e415dee0 con 0x7f07e40d0ef0 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07cc003520 con 0x7f07e40d0ef0 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3403502928 0 0) 0x7f07cc003520 con 0x7f07e40d0ef0 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 msgr2=0x7f07e415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 0x7f07e415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 >> v1:172.21.15.137:6789/0 conn(0x7f07e40d3e50 legacy=0x7f07e40d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e415e780 con 0x7f07e40d0ef0 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f07dc0019f0 con 0x7f07e40d0ef0 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 >> v1:172.21.15.31:6789/0 conn(0x7f07e40d0ef0 legacy=0x7f07e40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 0x7f07cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07cc004d00 0x7f07cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e415e4a0 con 0x7f07cc0039a0 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc00ba30 con 0x7f07cc004d00 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc00bbf0 con 0x7f07e40d18e0 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e3fff640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e37fe640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 0x7f07cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.101+0000 7f07e8dbb640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07cc004d00 0x7f07cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e3fff640 1 -- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 msgr2=0x7f07cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e3fff640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 0x7f07cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e3fff640 1 -- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07cc004d00 msgr2=0x7f07cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e3fff640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07cc004d00 0x7f07cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e3fff640 1 -- 172.21.15.31:0/2873885844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc003520 con 0x7f07cc0039a0 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e37fe640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40d18e0 0x7f07cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e3fff640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009a90 tx=0x7f07cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d4a5c30c4b095c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.102+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d802c680 con 0x7f07cc0039a0 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d802c7e0 con 0x7f07cc0039a0 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07e2ffd640 1 -- 172.21.15.31:0/2873885844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d802cb70 con 0x7f07cc0039a0 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07ea81e640 1 -- 172.21.15.31:0/2873885844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 msgr2=0x7f07cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07ea81e640 1 --2- 172.21.15.31:0/2873885844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009a90 tx=0x7f07cc0057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07e8dbb640 1 -- 172.21.15.31:0/2873885844 reap_dead start 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07ea81e640 1 -- 172.21.15.31:0/2873885844 shutdown_connections 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07ea81e640 1 -- 172.21.15.31:0/2873885844 >> 172.21.15.31:0/2873885844 conn(0x7f07e409c600 msgr2=0x7f07e40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07ea81e640 1 -- 172.21.15.31:0/2873885844 shutdown_connections 2024-01-19T04:56:01.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.103+0000 7f07ea81e640 1 -- 172.21.15.31:0/2873885844 wait complete. 2024-01-19T04:56:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.104+0000 7f07ea81e640 1 Processor -- start 2024-01-19T04:56:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.104+0000 7f07ea81e640 1 -- start start 2024-01-19T04:56:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07ea81e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07e40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07ea81e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40cc670 0x7f07e41f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07ea81e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e41f3610 0x7f07e40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07ea81e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e41f4070 con 0x7f07cc0039a0 2024-01-19T04:56:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07ea81e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e41f3ab0 con 0x7f07e41f3610 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07ea81e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e41f3d90 con 0x7f07e40cc670 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07e40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07e40b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39764/0 (socket says 172.21.15.31:39764) 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 -- 172.21.15.31:0/1460902731 learned_addr learned my addr 172.21.15.31:0/1460902731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 -- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40cc670 msgr2=0x7f07e41f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40cc670 0x7f07e41f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 -- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e41f3610 msgr2=0x7f07e40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e41f3610 0x7f07e40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.105+0000 7f07e3fff640 1 -- 172.21.15.31:0/1460902731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40c6f80 con 0x7f07cc0039a0 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.106+0000 7f07e3fff640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07e40b8b00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f07d8037040 tx=0x7f07d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.106+0000 7f07e0ff9640 1 -- 172.21.15.31:0/1460902731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8003810 con 0x7f07cc0039a0 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.106+0000 7f07e0ff9640 1 -- 172.21.15.31:0/1460902731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8003990 con 0x7f07cc0039a0 2024-01-19T04:56:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.106+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40c7260 con 0x7f07cc0039a0 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.106+0000 7f07e0ff9640 1 -- 172.21.15.31:0/1460902731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8003d40 con 0x7f07cc0039a0 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.106+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e40c7cc0 con 0x7f07cc0039a0 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.107+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e40c73f0 con 0x7f07cc0039a0 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.107+0000 7f07e0ff9640 1 -- 172.21.15.31:0/1460902731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f07d8034440 con 0x7f07cc0039a0 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.107+0000 7f07e0ff9640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07d006aed0 0x7f07d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.108+0000 7f07e0ff9640 1 -- 172.21.15.31:0/1460902731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f07d80ab5f0 con 0x7f07cc0039a0 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.108+0000 7f07e37fe640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07d006aed0 0x7f07d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.108+0000 7f07e37fe640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07d006aed0 0x7f07d006d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07d4007aa0 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.112+0000 7f07e0ff9640 1 -- 172.21.15.31:0/1460902731 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f07d80766a0 con 0x7f07cc0039a0 2024-01-19T04:56:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.288+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07e415dee0 con 0x7f07cc0039a0 2024-01-19T04:56:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.289+0000 7f07e0ff9640 1 -- 172.21.15.31:0/1460902731 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07d8079cc0 con 0x7f07cc0039a0 2024-01-19T04:56:01.289 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:01.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07d006aed0 msgr2=0x7f07d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07d006aed0 0x7f07d006d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07d4007aa0 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 msgr2=0x7f07e40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07e40b8b00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f07d8037040 tx=0x7f07d8004730 comp rx=0 tx=0).stop 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 shutdown_connections 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07d006aed0 0x7f07d006d390 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e41f3610 0x7f07e40c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07e40cc670 0x7f07e41f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 --2- 172.21.15.31:0/1460902731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc0039a0 0x7f07e40b8b00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 >> 172.21.15.31:0/1460902731 conn(0x7f07e409c600 msgr2=0x7f07e40b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 shutdown_connections 2024-01-19T04:56:01.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.291+0000 7f07ea81e640 1 -- 172.21.15.31:0/1460902731 wait complete. 2024-01-19T04:56:01.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b2f1d640 1 Processor -- start 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b2f1d640 1 -- start start 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b2f1d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98ac0d1c20 0x7f98ac0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b2f1d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f98ac0d3ea0 0x7f98ac0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b2f1d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac0cf160 con 0x7f98ac15d720 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b2f1d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac0ce8c0 con 0x7f98ac0d1c20 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b2f1d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac0ceba0 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98abfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98ac0d1c20 0x7f98ac0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b0cb9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f98ac15d720 0x7f98ac0d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58366/0 (socket says 172.21.15.31:58366) 2024-01-19T04:56:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.406+0000 7f98b0cb9640 1 -- 172.21.15.31:0/852225310 learned_addr learned my addr 172.21.15.31:0/852225310 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.407+0000 7f98abfff640 1 -- 172.21.15.31:0/852225310 >> v2:172.21.15.137:3300/0 conn(0x7f98ac0d3ea0 msgr2=0x7f98ac0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.407+0000 7f98b14ba640 1 --2- 172.21.15.31:0/852225310 >> v2:172.21.15.137:3300/0 conn(0x7f98ac0d3ea0 0x7f98ac0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.407+0000 7f98ab7fe640 1 -- 172.21.15.31:0/852225310 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090565814 0 0) 0x7f98ac0cf160 con 0x7f98ac15d720 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.407+0000 7f98abfff640 1 --2- 172.21.15.31:0/852225310 >> v2:172.21.15.137:3300/0 conn(0x7f98ac0d3ea0 0x7f98ac0b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.407+0000 7f98abfff640 1 -- 172.21.15.31:0/852225310 >> v1:172.21.15.31:6789/0 conn(0x7f98ac15d720 legacy=0x7f98ac0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.407+0000 7f98abfff640 1 -- 172.21.15.31:0/852225310 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac0cee80 con 0x7f98ac0d1c20 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.407+0000 7f98abfff640 1 --2- 172.21.15.31:0/852225310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d1c20 0x7f98ac0d0ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98ac1611a0 tx=0x7f98a002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6161cb0f315a621 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98ab7fe640 1 -- 172.21.15.31:0/852225310 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0031710 con 0x7f98ac0d1c20 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98ab7fe640 1 -- 172.21.15.31:0/852225310 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a002f050 con 0x7f98ac0d1c20 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98ab7fe640 1 -- 172.21.15.31:0/852225310 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0031a90 con 0x7f98ac0d1c20 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 -- 172.21.15.31:0/852225310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d1c20 msgr2=0x7f98ac0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/852225310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d1c20 0x7f98ac0d0ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98ac1611a0 tx=0x7f98a002b7b0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 -- 172.21.15.31:0/852225310 shutdown_connections 2024-01-19T04:56:01.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/852225310 >> v2:172.21.15.137:3300/0 conn(0x7f98ac0d3ea0 0x7f98ac0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/852225310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d1c20 0x7f98ac0d0ee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 -- 172.21.15.31:0/852225310 >> 172.21.15.31:0/852225310 conn(0x7f98ac0d3310 msgr2=0x7f98ac0d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 -- 172.21.15.31:0/852225310 shutdown_connections 2024-01-19T04:56:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.408+0000 7f98b2f1d640 1 -- 172.21.15.31:0/852225310 wait complete. 2024-01-19T04:56:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.409+0000 7f98b2f1d640 1 Processor -- start 2024-01-19T04:56:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.409+0000 7f98b2f1d640 1 -- start start 2024-01-19T04:56:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98b2f1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98ac0d1c20 0x7f98ac1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98b2f1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d3ea0 0x7f98ac0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98b2f1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98ac15d720 0x7f98ac0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98b2f1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c8de0 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98b2f1d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0b5fb0 con 0x7f98ac0d1c20 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98b2f1d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c8ff0 con 0x7f98ac15d720 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d3ea0 0x7f98ac0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d3ea0 0x7f98ac0c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39772/0 (socket says 172.21.15.31:39772) 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.410+0000 7f98abfff640 1 -- 172.21.15.31:0/2554875801 learned_addr learned my addr 172.21.15.31:0/2554875801 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98b0cb9640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98ac0d1c20 0x7f98ac1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98b14ba640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98ac15d720 0x7f98ac0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98abfff640 1 -- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98ac15d720 msgr2=0x7f98ac0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98abfff640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98ac15d720 0x7f98ac0c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98abfff640 1 -- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98ac0d1c20 msgr2=0x7f98ac1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98abfff640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98ac0d1c20 0x7f98ac1593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98abfff640 1 -- 172.21.15.31:0/2554875801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a0009400 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98b14ba640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98ac15d720 0x7f98ac0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98b0cb9640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98ac0d1c20 0x7f98ac1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98abfff640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d3ea0 0x7f98ac0c9880 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f98a0000c00 tx=0x7f98a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98a97fa640 1 -- 172.21.15.31:0/2554875801 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0033a50 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98a97fa640 1 -- 172.21.15.31:0/2554875801 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0033bb0 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98a97fa640 1 -- 172.21.15.31:0/2554875801 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a003c400 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac0b6250 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.411+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac0c92d0 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.412+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac0b6770 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.413+0000 7f98a97fa640 1 -- 172.21.15.31:0/2554875801 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f98a003a040 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.413+0000 7f98a97fa640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f989806aed0 0x7f989806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.413+0000 7f98a97fa640 1 -- 172.21.15.31:0/2554875801 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f98a00aae10 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.413+0000 7f98b0cb9640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f989806aed0 0x7f989806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.414+0000 7f98b0cb9640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f989806aed0 0x7f989806d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f98a40012e0 tx=0x7f98a4005b90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.417+0000 7f98a97fa640 1 -- 172.21.15.31:0/2554875801 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f98ac0b6770 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.597+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98ac0b6770 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.597+0000 7f98a97fa640 1 -- 172.21.15.31:0/2554875801 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98ac0b6770 con 0x7f98ac0d3ea0 2024-01-19T04:56:01.598 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:01.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f989806aed0 msgr2=0x7f989806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f989806aed0 0x7f989806d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f98a40012e0 tx=0x7f98a4005b90 comp rx=0 tx=0).stop 2024-01-19T04:56:01.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d3ea0 msgr2=0x7f98ac0c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d3ea0 0x7f98ac0c9880 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f98a0000c00 tx=0x7f98a00046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 shutdown_connections 2024-01-19T04:56:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f989806aed0 0x7f989806d390 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98ac15d720 0x7f98ac0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98ac0d3ea0 0x7f98ac0c9880 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 --2- 172.21.15.31:0/2554875801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98ac0d1c20 0x7f98ac1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 >> 172.21.15.31:0/2554875801 conn(0x7f98ac0d3310 msgr2=0x7f98ac0cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 shutdown_connections 2024-01-19T04:56:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.599+0000 7f98b2f1d640 1 -- 172.21.15.31:0/2554875801 wait complete. 2024-01-19T04:56:01.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:01.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.712+0000 7fad39be3640 1 Processor -- start 2024-01-19T04:56:01.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.712+0000 7fad39be3640 1 -- start start 2024-01-19T04:56:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad39be3640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fad3415d3a0 0x7fad3406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad39be3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad34161860 0x7fad34167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad39be3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340cf9f0 con 0x7fad340cf570 2024-01-19T04:56:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad39be3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340cfcd0 con 0x7fad34161860 2024-01-19T04:56:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad39be3640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340cffb0 con 0x7fad3415d3a0 2024-01-19T04:56:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad33fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad34161860 0x7fad34167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad32ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fad340cf570 0x7fad34161150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58382/0 (socket says 172.21.15.31:58382) 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad32ffd640 1 -- 172.21.15.31:0/1348468581 learned_addr learned my addr 172.21.15.31:0/1348468581 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad33fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad34161860 0x7fad34167480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39774/0 (socket says 172.21.15.31:39774) 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad337fe640 1 --2- 172.21.15.31:0/1348468581 >> v2:172.21.15.137:3300/0 conn(0x7fad3415d3a0 0x7fad3406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad33fff640 1 -- 172.21.15.31:0/1348468581 >> v2:172.21.15.137:3300/0 conn(0x7fad3415d3a0 msgr2=0x7fad3406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad33fff640 1 --2- 172.21.15.31:0/1348468581 >> v2:172.21.15.137:3300/0 conn(0x7fad3415d3a0 0x7fad3406c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad33fff640 1 -- 172.21.15.31:0/1348468581 >> v1:172.21.15.31:6789/0 conn(0x7fad340cf570 legacy=0x7fad34161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad33fff640 1 -- 172.21.15.31:0/1348468581 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340d0290 con 0x7fad34161860 2024-01-19T04:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.713+0000 7fad327fc640 1 -- 172.21.15.31:0/1348468581 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052168911 0 0) 0x7fad340cf9f0 con 0x7fad340cf570 2024-01-19T04:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.714+0000 7fad33fff640 1 --2- 172.21.15.31:0/1348468581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad34161860 0x7fad34167480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fad3406dd90 tx=0x7fad2c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0be67eb2660952 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.714+0000 7fad327fc640 1 -- 172.21.15.31:0/1348468581 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c02cbf0 con 0x7fad34161860 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.714+0000 7fad327fc640 1 -- 172.21.15.31:0/1348468581 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c02cd50 con 0x7fad34161860 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.714+0000 7fad327fc640 1 -- 172.21.15.31:0/1348468581 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c034660 con 0x7fad34161860 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 -- 172.21.15.31:0/1348468581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad34161860 msgr2=0x7fad34167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 --2- 172.21.15.31:0/1348468581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad34161860 0x7fad34167480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fad3406dd90 tx=0x7fad2c02be40 comp rx=0 tx=0).stop 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 -- 172.21.15.31:0/1348468581 shutdown_connections 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 --2- 172.21.15.31:0/1348468581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad34161860 0x7fad34167480 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 --2- 172.21.15.31:0/1348468581 >> v2:172.21.15.137:3300/0 conn(0x7fad3415d3a0 0x7fad3406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 -- 172.21.15.31:0/1348468581 >> 172.21.15.31:0/1348468581 conn(0x7fad340d1100 msgr2=0x7fad340d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 -- 172.21.15.31:0/1348468581 shutdown_connections 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.715+0000 7fad39be3640 1 -- 172.21.15.31:0/1348468581 wait complete. 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.716+0000 7fad39be3640 1 Processor -- start 2024-01-19T04:56:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.716+0000 7fad39be3640 1 -- start start 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad39be3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad340cf570 0x7fad34152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad39be3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad3415d3a0 0x7fad340b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad39be3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fad34161860 0x7fad3415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad39be3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b5ac0 con 0x7fad340cf570 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad39be3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b5da0 con 0x7fad3415d3a0 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad39be3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b6080 con 0x7fad34161860 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad340cf570 0x7fad34152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad340cf570 0x7fad34152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39790/0 (socket says 172.21.15.31:39790) 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 -- 172.21.15.31:0/2898175845 learned_addr learned my addr 172.21.15.31:0/2898175845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 -- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fad34161860 msgr2=0x7fad3415a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fad34161860 0x7fad3415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 -- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad3415d3a0 msgr2=0x7fad340b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad3415d3a0 0x7fad340b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 -- 172.21.15.31:0/2898175845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c009a70 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad337fe640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad340cf570 0x7fad34152480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fad2400d800 tx=0x7fad2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.717+0000 7fad1bfff640 1 -- 172.21.15.31:0/2898175845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24018070 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.718+0000 7fad1bfff640 1 -- 172.21.15.31:0/2898175845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad24005990 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.718+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad340b6360 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.718+0000 7fad1bfff640 1 -- 172.21.15.31:0/2898175845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24005b40 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.718+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad340b5790 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.719+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad340b6880 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.719+0000 7fad1bfff640 1 -- 172.21.15.31:0/2898175845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fad2400b6c0 con 0x7fad340cf570 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.719+0000 7fad1bfff640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fad2006aed0 0x7fad2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.719+0000 7fad1bfff640 1 -- 172.21.15.31:0/2898175845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fad24089650 con 0x7fad340cf570 2024-01-19T04:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.720+0000 7fad32ffd640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fad2006aed0 0x7fad2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.720+0000 7fad32ffd640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fad2006aed0 0x7fad2006d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fad28007fd0 tx=0x7fad28007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.724+0000 7fad1bfff640 1 -- 172.21.15.31:0/2898175845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fad340b6880 con 0x7fad340cf570 2024-01-19T04:56:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.916+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fad340b6880 con 0x7fad340cf570 2024-01-19T04:56:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.916+0000 7fad1bfff640 1 -- 172.21.15.31:0/2898175845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fad340b6880 con 0x7fad340cf570 2024-01-19T04:56:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fad2006aed0 msgr2=0x7fad2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fad2006aed0 0x7fad2006d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fad28007fd0 tx=0x7fad28007920 comp rx=0 tx=0).stop 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad340cf570 msgr2=0x7fad34152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad340cf570 0x7fad34152480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fad2400d800 tx=0x7fad2400dcc0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 shutdown_connections 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fad2006aed0 0x7fad2006d390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fad34161860 0x7fad3415a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad3415d3a0 0x7fad340b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 --2- 172.21.15.31:0/2898175845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad340cf570 0x7fad34152480 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:01.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 >> 172.21.15.31:0/2898175845 conn(0x7fad340d1100 msgr2=0x7fad340add30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:01.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 shutdown_connections 2024-01-19T04:56:01.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:01.918+0000 7fad39be3640 1 -- 172.21.15.31:0/2898175845 wait complete. 2024-01-19T04:56:01.930 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:56:01.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:01.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-01-19T04:56:01.957 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:01.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:56:01.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:01.958 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-19T04:56:02.012 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:56:02.012 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:02.031 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:56:02.032 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:02.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.191+0000 7f18b6901640 1 Processor -- start 2024-01-19T04:56:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.191+0000 7f18b6901640 1 -- start start 2024-01-19T04:56:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18b6901640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18b015d300 0x7f18b006ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18b6901640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b0071df0 0x7f18b00aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18b6901640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00b3510 con 0x7f18b00b3030 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18b6901640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00b37f0 con 0x7f18b015d300 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18b6901640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00b3ad0 con 0x7f18b0071df0 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18affff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18b015d300 0x7f18b006ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18affff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18b015d300 0x7f18b006ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39800/0 (socket says 172.21.15.31:39800) 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18affff640 1 -- 172.21.15.31:0/1300434972 learned_addr learned my addr 172.21.15.31:0/1300434972 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18b4e9e640 1 --2- 172.21.15.31:0/1300434972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b0071df0 0x7f18b00aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18affff640 1 -- 172.21.15.31:0/1300434972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b0071df0 msgr2=0x7f18b00aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18affff640 1 --2- 172.21.15.31:0/1300434972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b0071df0 0x7f18b00aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18affff640 1 -- 172.21.15.31:0/1300434972 >> v1:172.21.15.137:6789/0 conn(0x7f18b00b3030 legacy=0x7f18b0160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18affff640 1 -- 172.21.15.31:0/1300434972 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b006a650 con 0x7f18b015d300 2024-01-19T04:56:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.192+0000 7f18b4e9e640 1 --2- 172.21.15.31:0/1300434972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b0071df0 0x7f18b00aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.193+0000 7f18affff640 1 --2- 172.21.15.31:0/1300434972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b015d300 0x7f18b006ab70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009870 tx=0x7f18a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4529f39a79d276 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.193+0000 7f18aeffd640 1 -- 172.21.15.31:0/1300434972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0031640 con 0x7f18b015d300 2024-01-19T04:56:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.193+0000 7f18aeffd640 1 -- 172.21.15.31:0/1300434972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a00317a0 con 0x7f18b015d300 2024-01-19T04:56:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.193+0000 7f18aeffd640 1 -- 172.21.15.31:0/1300434972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0031b30 con 0x7f18b015d300 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 -- 172.21.15.31:0/1300434972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b015d300 msgr2=0x7f18b006ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 --2- 172.21.15.31:0/1300434972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b015d300 0x7f18b006ab70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009870 tx=0x7f18a002b8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 -- 172.21.15.31:0/1300434972 shutdown_connections 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 --2- 172.21.15.31:0/1300434972 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b0071df0 0x7f18b00aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 --2- 172.21.15.31:0/1300434972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b015d300 0x7f18b006ab70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 -- 172.21.15.31:0/1300434972 >> 172.21.15.31:0/1300434972 conn(0x7f18b006d750 msgr2=0x7f18b006db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 -- 172.21.15.31:0/1300434972 shutdown_connections 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.194+0000 7f18b6901640 1 -- 172.21.15.31:0/1300434972 wait complete. 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.195+0000 7f18b6901640 1 Processor -- start 2024-01-19T04:56:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.195+0000 7f18b6901640 1 -- start start 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18b6901640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b0071df0 0x7f18b01f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18b6901640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b00b3030 0x7f18b0159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18b6901640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b015d300 0x7f18b015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18b6901640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b015a970 con 0x7f18b0071df0 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18b6901640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00cc9f0 con 0x7f18b00b3030 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18b6901640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00cccd0 con 0x7f18b015d300 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b0071df0 0x7f18b01f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b0071df0 0x7f18b01f30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39808/0 (socket says 172.21.15.31:39808) 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 -- 172.21.15.31:0/1503432844 learned_addr learned my addr 172.21.15.31:0/1503432844 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18b4e9e640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b015d300 0x7f18b015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18af7fe640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b00b3030 0x7f18b0159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 -- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b015d300 msgr2=0x7f18b015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b015d300 0x7f18b015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 -- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b00b3030 msgr2=0x7f18b0159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b00b3030 0x7f18b0159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 -- 172.21.15.31:0/1503432844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00094d0 con 0x7f18b0071df0 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.196+0000 7f18affff640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b0071df0 0x7f18b01f30a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009870 tx=0x7f18a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.197+0000 7f18acff9640 1 -- 172.21.15.31:0/1503432844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a003ac60 con 0x7f18b0071df0 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.197+0000 7f18acff9640 1 -- 172.21.15.31:0/1503432844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a003ade0 con 0x7f18b0071df0 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.197+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b00ccfb0 con 0x7f18b0071df0 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.197+0000 7f18acff9640 1 -- 172.21.15.31:0/1503432844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0044630 con 0x7f18b0071df0 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.197+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b01f35e0 con 0x7f18b0071df0 2024-01-19T04:56:02.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.198+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b00cd4d0 con 0x7f18b0071df0 2024-01-19T04:56:02.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.198+0000 7f18acff9640 1 -- 172.21.15.31:0/1503432844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f18a002f050 con 0x7f18b0071df0 2024-01-19T04:56:02.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.198+0000 7f18acff9640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f189c06af00 0x7f189c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.199+0000 7f18acff9640 1 -- 172.21.15.31:0/1503432844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f18a00ab6f0 con 0x7f18b0071df0 2024-01-19T04:56:02.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.199+0000 7f18af7fe640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f189c06af00 0x7f189c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.199+0000 7f18af7fe640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f189c06af00 0x7f189c06d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18b01540a0 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.203+0000 7f18acff9640 1 -- 172.21.15.31:0/1503432844 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f18b00cd4d0 con 0x7f18b0071df0 2024-01-19T04:56:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.385+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18b00cd4d0 con 0x7f18b0071df0 2024-01-19T04:56:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.386+0000 7f18acff9640 1 -- 172.21.15.31:0/1503432844 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16720 (secure 0 0 0) 0x7f18b00cd4d0 con 0x7f18b0071df0 2024-01-19T04:56:02.387 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:02.387 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:01.578429+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f189c06af00 msgr2=0x7f189c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f189c06af00 0x7f189c06d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18b01540a0 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b0071df0 msgr2=0x7f18b01f30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b0071df0 0x7f18b01f30a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009870 tx=0x7f18a00046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 shutdown_connections 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f189c06af00 0x7f189c06d3c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18b015d300 0x7f18b015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b00b3030 0x7f18b0159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 --2- 172.21.15.31:0/1503432844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18b0071df0 0x7f18b01f30a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 >> 172.21.15.31:0/1503432844 conn(0x7f18b006d750 msgr2=0x7f18b00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 shutdown_connections 2024-01-19T04:56:02.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.389+0000 7f18b6901640 1 -- 172.21.15.31:0/1503432844 wait complete. 2024-01-19T04:56:02.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:02.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.505+0000 7fa2c4a23640 1 Processor -- start 2024-01-19T04:56:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.505+0000 7fa2c4a23640 1 -- start start 2024-01-19T04:56:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.506+0000 7fa2c4a23640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa2c00add00 0x7fa2c00cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.506+0000 7fa2c4a23640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00b0a70 con 0x7fa2c00ccc70 2024-01-19T04:56:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.506+0000 7fa2c4a23640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00b18d0 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.506+0000 7fa2c4a23640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c00cd080 con 0x7fa2c00add00 2024-01-19T04:56:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.506+0000 7fa2bdd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa2c00add00 0x7fa2c00cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.506+0000 7fa2bed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa2c00ccc70 0x7fa2c00b1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58396/0 (socket says 172.21.15.31:58396) 2024-01-19T04:56:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.506+0000 7fa2bed9d640 1 -- 172.21.15.31:0/3879392517 learned_addr learned my addr 172.21.15.31:0/3879392517 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.507+0000 7fa2bdd9b640 1 -- 172.21.15.31:0/3879392517 >> v1:172.21.15.137:6789/0 conn(0x7fa2c00cf6f0 legacy=0x7fa2c00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.507+0000 7fa2bdd9b640 1 -- 172.21.15.31:0/3879392517 >> v1:172.21.15.31:6789/0 conn(0x7fa2c00ccc70 legacy=0x7fa2c00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.507+0000 7fa2bd59a640 1 -- 172.21.15.31:0/3879392517 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843080752 0 0) 0x7fa2c00b0a70 con 0x7fa2c00ccc70 2024-01-19T04:56:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.507+0000 7fa2bdd9b640 1 -- 172.21.15.31:0/3879392517 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00ae160 con 0x7fa2c00add00 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.507+0000 7fa2bdd9b640 1 --2- 172.21.15.31:0/3879392517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00add00 0x7fa2c00cc730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40097a0 tx=0x7fa2b402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb892be907224d9e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.507+0000 7fa2bd59a640 1 -- 172.21.15.31:0/3879392517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4030610 con 0x7fa2c00add00 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2bd59a640 1 -- 172.21.15.31:0/3879392517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4030770 con 0x7fa2c00add00 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2bd59a640 1 -- 172.21.15.31:0/3879392517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4030b00 con 0x7fa2c00add00 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2c4a23640 1 -- 172.21.15.31:0/3879392517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00add00 msgr2=0x7fa2c00cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/3879392517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00add00 0x7fa2c00cc730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40097a0 tx=0x7fa2b402b960 comp rx=0 tx=0).stop 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2c4a23640 1 -- 172.21.15.31:0/3879392517 shutdown_connections 2024-01-19T04:56:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/3879392517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00add00 0x7fa2c00cc730 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2c4a23640 1 -- 172.21.15.31:0/3879392517 >> 172.21.15.31:0/3879392517 conn(0x7fa2c008afe0 msgr2=0x7fa2c00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:02.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2c4a23640 1 -- 172.21.15.31:0/3879392517 shutdown_connections 2024-01-19T04:56:02.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.508+0000 7fa2c4a23640 1 -- 172.21.15.31:0/3879392517 wait complete. 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.509+0000 7fa2c4a23640 1 Processor -- start 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.509+0000 7fa2c4a23640 1 -- start start 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2c4a23640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c00add00 0x7fa2c006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2c4a23640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa2c00ccc70 0x7fa2c006c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2c4a23640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00cf6f0 0x7fa2c00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2c4a23640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c006c6a0 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2c4a23640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00b6700 con 0x7fa2c00add00 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2c4a23640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c006b6c0 con 0x7fa2c00ccc70 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00cf6f0 0x7fa2c00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00cf6f0 0x7fa2c00b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39836/0 (socket says 172.21.15.31:39836) 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.510+0000 7fa2bed9d640 1 -- 172.21.15.31:0/1468757142 learned_addr learned my addr 172.21.15.31:0/1468757142 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bdd9b640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa2c00ccc70 0x7fa2c006c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2be59c640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c00add00 0x7fa2c006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bed9d640 1 -- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa2c00ccc70 msgr2=0x7fa2c006c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bed9d640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa2c00ccc70 0x7fa2c006c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bed9d640 1 -- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c00add00 msgr2=0x7fa2c006d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bed9d640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c00add00 0x7fa2c006d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bed9d640 1 -- 172.21.15.31:0/1468757142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4009400 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bdd9b640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa2c00ccc70 0x7fa2c006c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2be59c640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c00add00 0x7fa2c006d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2bed9d640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00cf6f0 0x7fa2c00b61c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c7a0 tx=0x7fa2b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2aeffd640 1 -- 172.21.15.31:0/1468757142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000a5e0 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2aeffd640 1 -- 172.21.15.31:0/1468757142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b000a740 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c00b6940 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2aeffd640 1 -- 172.21.15.31:0/1468757142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000ab30 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.511+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c006b960 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.512+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0074900 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.512+0000 7fa2aeffd640 1 -- 172.21.15.31:0/1468757142 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa2b000ac90 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.513+0000 7fa2aeffd640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa2a006aed0 0x7fa2a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.513+0000 7fa2aeffd640 1 -- 172.21.15.31:0/1468757142 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa2b0088e40 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.513+0000 7fa2be59c640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa2a006aed0 0x7fa2a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.514+0000 7fa2be59c640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa2a006aed0 0x7fa2a006d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8007fd0 tx=0x7fa2b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.517+0000 7fa2aeffd640 1 -- 172.21.15.31:0/1468757142 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa2b0053c90 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.698+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2c00cf4d0 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.698+0000 7fa2aeffd640 1 -- 172.21.15.31:0/1468757142 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2b0018020 con 0x7fa2c00cf6f0 2024-01-19T04:56:02.699 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa2a006aed0 msgr2=0x7fa2a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa2a006aed0 0x7fa2a006d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8007fd0 tx=0x7fa2b8009040 comp rx=0 tx=0).stop 2024-01-19T04:56:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00cf6f0 msgr2=0x7fa2c00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00cf6f0 0x7fa2c00b61c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c7a0 tx=0x7fa2b000cc60 comp rx=0 tx=0).stop 2024-01-19T04:56:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 shutdown_connections 2024-01-19T04:56:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa2a006aed0 0x7fa2a006d390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c00cf6f0 0x7fa2c00b61c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa2c00ccc70 0x7fa2c006c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 --2- 172.21.15.31:0/1468757142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c00add00 0x7fa2c006d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 >> 172.21.15.31:0/1468757142 conn(0x7fa2c008afe0 msgr2=0x7fa2c00af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 shutdown_connections 2024-01-19T04:56:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.701+0000 7fa2c4a23640 1 -- 172.21.15.31:0/1468757142 wait complete. 2024-01-19T04:56:02.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb800aa2640 1 Processor -- start 2024-01-19T04:56:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb800aa2640 1 -- start start 2024-01-19T04:56:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb800aa2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7fc0716f0 0x7fb7fc15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb800aa2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb7fc15b510 0x7fb7fc06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb800aa2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc071c20 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb800aa2640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc071f00 con 0x7fb7fc0716f0 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb800aa2640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc0721e0 con 0x7fb7fc15b510 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7f3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7fc0716f0 0x7fb7fc15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7f3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7fc0716f0 0x7fb7fc15afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39846/0 (socket says 172.21.15.31:39846) 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7f3fff640 1 -- 172.21.15.31:0/1905834008 learned_addr learned my addr 172.21.15.31:0/1905834008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7fad9d640 1 --2- 172.21.15.31:0/1905834008 >> v2:172.21.15.137:3300/0 conn(0x7fb7fc15b510 0x7fb7fc06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7f3fff640 1 -- 172.21.15.31:0/1905834008 >> v2:172.21.15.137:3300/0 conn(0x7fb7fc15b510 msgr2=0x7fb7fc06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7f3fff640 1 --2- 172.21.15.31:0/1905834008 >> v2:172.21.15.137:3300/0 conn(0x7fb7fc15b510 0x7fb7fc06d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7f3fff640 1 -- 172.21.15.31:0/1905834008 >> v1:172.21.15.137:6789/0 conn(0x7fb7fc0cb2e0 legacy=0x7fb7fc15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.816+0000 7fb7f3fff640 1 -- 172.21.15.31:0/1905834008 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc15aab0 con 0x7fb7fc0716f0 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.817+0000 7fb7f3fff640 1 --2- 172.21.15.31:0/1905834008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0716f0 0x7fb7fc15afd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0b3c60 tx=0x7fb7e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd27c726886b13d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.817+0000 7fb7f9d9b640 1 -- 172.21.15.31:0/1905834008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4031870 con 0x7fb7fc0716f0 2024-01-19T04:56:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.817+0000 7fb7f9d9b640 1 -- 172.21.15.31:0/1905834008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40319d0 con 0x7fb7fc0716f0 2024-01-19T04:56:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.817+0000 7fb7f9d9b640 1 -- 172.21.15.31:0/1905834008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4031d60 con 0x7fb7fc0716f0 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 -- 172.21.15.31:0/1905834008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0716f0 msgr2=0x7fb7fc15afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 --2- 172.21.15.31:0/1905834008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0716f0 0x7fb7fc15afd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0b3c60 tx=0x7fb7e402b910 comp rx=0 tx=0).stop 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 -- 172.21.15.31:0/1905834008 shutdown_connections 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 --2- 172.21.15.31:0/1905834008 >> v2:172.21.15.137:3300/0 conn(0x7fb7fc15b510 0x7fb7fc06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 --2- 172.21.15.31:0/1905834008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0716f0 0x7fb7fc15afd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 -- 172.21.15.31:0/1905834008 >> 172.21.15.31:0/1905834008 conn(0x7fb7fc06aa00 msgr2=0x7fb7fc06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 -- 172.21.15.31:0/1905834008 shutdown_connections 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.818+0000 7fb800aa2640 1 -- 172.21.15.31:0/1905834008 wait complete. 2024-01-19T04:56:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.819+0000 7fb800aa2640 1 Processor -- start 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.819+0000 7fb800aa2640 1 -- start start 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb800aa2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7fc0716f0 0x7fb7fc0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb800aa2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0cb2e0 0x7fb7fc14e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb800aa2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7fc15b510 0x7fb7fc14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb800aa2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc14e5b0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb800aa2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc150040 con 0x7fb7fc15b510 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb800aa2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc14f4c0 con 0x7fb7fc0716f0 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0cb2e0 0x7fb7fc14e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0cb2e0 0x7fb7fc14e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39852/0 (socket says 172.21.15.31:39852) 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 -- 172.21.15.31:0/2858325874 learned_addr learned my addr 172.21.15.31:0/2858325874 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7fad9d640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7fc15b510 0x7fb7fc14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7fa59c640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7fc0716f0 0x7fb7fc0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 -- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7fc0716f0 msgr2=0x7fb7fc0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7fc0716f0 0x7fb7fc0afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 -- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7fc15b510 msgr2=0x7fb7fc14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7fc15b510 0x7fb7fc14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 -- 172.21.15.31:0/2858325874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e40094d0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7fa59c640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7fc0716f0 0x7fb7fc0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7fad9d640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7fc15b510 0x7fb7fc14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.820+0000 7fb7f3fff640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0cb2e0 0x7fb7fc14e940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000c00 tx=0x7fb7e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.821+0000 7fb7f2ffd640 1 -- 172.21.15.31:0/2858325874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4031840 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.821+0000 7fb7f2ffd640 1 -- 172.21.15.31:0/2858325874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40443f0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.821+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc14f7a0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.821+0000 7fb7f2ffd640 1 -- 172.21.15.31:0/2858325874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40447a0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.821+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc14fd60 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.822+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc074900 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.822+0000 7fb7f2ffd640 1 -- 172.21.15.31:0/2858325874 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb7e404d450 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.823+0000 7fb7f2ffd640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb7dc06aed0 0x7fb7dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:02.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.823+0000 7fb7f2ffd640 1 -- 172.21.15.31:0/2858325874 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb7e402d040 con 0x7fb7fc0cb2e0 2024-01-19T04:56:02.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.823+0000 7fb7fa59c640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb7dc06aed0 0x7fb7dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:02.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.823+0000 7fb7fa59c640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb7dc06aed0 0x7fb7dc06d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec001750 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:02.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:02.827+0000 7fb7f2ffd640 1 -- 172.21.15.31:0/2858325874 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb7e4076ce0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.004+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7fc0b31d0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.005+0000 7fb7f2ffd640 1 -- 172.21.15.31:0/2858325874 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7e40350a0 con 0x7fb7fc0cb2e0 2024-01-19T04:56:03.005 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:03.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb7dc06aed0 msgr2=0x7fb7dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb7dc06aed0 0x7fb7dc06d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec001750 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0cb2e0 msgr2=0x7fb7fc14e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0cb2e0 0x7fb7fc14e940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000c00 tx=0x7fb7e40046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 shutdown_connections 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb7dc06aed0 0x7fb7dc06d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7fc15b510 0x7fb7fc14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7fc0cb2e0 0x7fb7fc14e940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 --2- 172.21.15.31:0/2858325874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7fc0716f0 0x7fb7fc0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 >> 172.21.15.31:0/2858325874 conn(0x7fb7fc06aa00 msgr2=0x7fb7fc06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 shutdown_connections 2024-01-19T04:56:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.007+0000 7fb800aa2640 1 -- 172.21.15.31:0/2858325874 wait complete. 2024-01-19T04:56:03.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:03.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.121+0000 7f3ad858e640 1 Processor -- start 2024-01-19T04:56:03.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.121+0000 7f3ad858e640 1 -- start start 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad858e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3ad00d18e0 0x7f3ad015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad858e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad015dee0 con 0x7f3ad00d3e50 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad858e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad015e1c0 con 0x7f3ad00d0ef0 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad858e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad015e4a0 con 0x7f3ad00d18e0 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad5b29640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3ad00d3e50 0x7f3ad00d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58402/0 (socket says 172.21.15.31:58402) 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad5b29640 1 -- 172.21.15.31:0/174231007 learned_addr learned my addr 172.21.15.31:0/174231007 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad632a640 1 --2- 172.21.15.31:0/174231007 >> v2:172.21.15.137:3300/0 conn(0x7f3ad00d18e0 0x7f3ad015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294878568 0 0) 0x7f3ad015dee0 con 0x7f3ad00d3e50 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.122+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab8003520 con 0x7f3ad00d3e50 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1397702205 0 0) 0x7f3ab8003520 con 0x7f3ad00d3e50 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 >> v2:172.21.15.137:3300/0 conn(0x7f3ad00d18e0 msgr2=0x7f3ad015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 --2- 172.21.15.31:0/174231007 >> v2:172.21.15.137:3300/0 conn(0x7f3ad00d18e0 0x7f3ad015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 >> v1:172.21.15.137:6789/0 conn(0x7f3ad00d0ef0 legacy=0x7f3ad00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad015e780 con 0x7f3ad00d3e50 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f3ac4001670 con 0x7f3ad00d3e50 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 >> v1:172.21.15.31:6789/0 conn(0x7f3ad00d3e50 legacy=0x7f3ad00d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab8003970 0x7f3ab8003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8004370 0x7f3ab8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ab8005040 0x7f3ab800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad015e4a0 con 0x7f3ab8003970 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab800bfa0 con 0x7f3ab8004370 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab800c160 con 0x7f3ab8005040 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad632a640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab8003970 0x7f3ab8003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad5b29640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8004370 0x7f3ab8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad6b2b640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ab8005040 0x7f3ab800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad632a640 1 -- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ab8005040 msgr2=0x7f3ab800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad632a640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ab8005040 0x7f3ab800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad632a640 1 -- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8004370 msgr2=0x7f3ab8004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad632a640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8004370 0x7f3ab8004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad632a640 1 -- 172.21.15.31:0/174231007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8003520 con 0x7f3ab8003970 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.123+0000 7f3ad6b2b640 1 -- 172.21.15.31:0/174231007 reap_dead start 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.124+0000 7f3ad632a640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab8003970 0x7f3ab8003d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009fc0 tx=0x7f3ab80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480b47cc19017577 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.124+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac002cc20 con 0x7f3ab8003970 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.124+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac000f040 con 0x7f3ab8003970 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.124+0000 7f3ad5328640 1 -- 172.21.15.31:0/174231007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0034440 con 0x7f3ab8003970 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.125+0000 7f3ad858e640 1 -- 172.21.15.31:0/174231007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab8003970 msgr2=0x7f3ab8003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.125+0000 7f3ad858e640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab8003970 0x7f3ab8003d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009fc0 tx=0x7f3ab80057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.125+0000 7f3ad858e640 1 -- 172.21.15.31:0/174231007 shutdown_connections 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.125+0000 7f3ad858e640 1 --2- 172.21.15.31:0/174231007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab8003970 0x7f3ab8003d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.125+0000 7f3ad858e640 1 -- 172.21.15.31:0/174231007 >> 172.21.15.31:0/174231007 conn(0x7f3ad009c600 msgr2=0x7f3ad00d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.125+0000 7f3ad858e640 1 -- 172.21.15.31:0/174231007 shutdown_connections 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.125+0000 7f3ad858e640 1 -- 172.21.15.31:0/174231007 wait complete. 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.126+0000 7f3ad858e640 1 Processor -- start 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.126+0000 7f3ad858e640 1 -- start start 2024-01-19T04:56:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad858e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8003970 0x7f3ad0150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad858e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ad00caae0 0x7f3ad00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad858e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ad015fb00 0x7f3ad015ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad858e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00c8c40 con 0x7f3ad015fb00 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad858e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00c8390 con 0x7f3ab8003970 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad858e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01604c0 con 0x7f3ad00caae0 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ad015fb00 0x7f3ad015ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ad015fb00 0x7f3ad015ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39884/0 (socket says 172.21.15.31:39884) 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 -- 172.21.15.31:0/2025631014 learned_addr learned my addr 172.21.15.31:0/2025631014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad632a640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8003970 0x7f3ad0150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad5b29640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ad00caae0 0x7f3ad00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 -- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ad00caae0 msgr2=0x7f3ad00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ad00caae0 0x7f3ad00caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 -- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8003970 msgr2=0x7f3ad0150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8003970 0x7f3ad0150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 -- 172.21.15.31:0/2025631014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00cb860 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad632a640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8003970 0x7f3ad0150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3ad6b2b640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ad015fb00 0x7f3ad015ff80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3ad008cc70 tx=0x7f3ac800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3aceffd640 1 -- 172.21.15.31:0/2025631014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80050d0 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.127+0000 7f3aceffd640 1 -- 172.21.15.31:0/2025631014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac8005250 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.128+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad01f3380 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.128+0000 7f3aceffd640 1 -- 172.21.15.31:0/2025631014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8007030 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.128+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad00cb580 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.128+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad00c8d80 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.129+0000 7f3aceffd640 1 -- 172.21.15.31:0/2025631014 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3ac8007190 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.129+0000 7f3aceffd640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3abc06aed0 0x7f3abc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.129+0000 7f3aceffd640 1 -- 172.21.15.31:0/2025631014 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3ac80888c0 con 0x7f3ad015fb00 2024-01-19T04:56:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.130+0000 7f3ad632a640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3abc06aed0 0x7f3abc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.130+0000 7f3ad632a640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3abc06aed0 0x7f3abc06d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0036000 tx=0x7f3ac002e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.135+0000 7f3aceffd640 1 -- 172.21.15.31:0/2025631014 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3ad00c8d80 con 0x7f3ad015fb00 2024-01-19T04:56:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.309+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ad0074900 con 0x7f3ad015fb00 2024-01-19T04:56:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.310+0000 7f3aceffd640 1 -- 172.21.15.31:0/2025631014 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ac80190a0 con 0x7f3ad015fb00 2024-01-19T04:56:03.310 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3abc06aed0 msgr2=0x7f3abc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3abc06aed0 0x7f3abc06d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0036000 tx=0x7f3ac002e000 comp rx=0 tx=0).stop 2024-01-19T04:56:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ad015fb00 msgr2=0x7f3ad015ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ad015fb00 0x7f3ad015ff80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3ad008cc70 tx=0x7f3ac800fc00 comp rx=0 tx=0).stop 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 shutdown_connections 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3abc06aed0 0x7f3abc06d390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ad015fb00 0x7f3ad015ff80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3ad00caae0 0x7f3ad00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 --2- 172.21.15.31:0/2025631014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ab8003970 0x7f3ad0150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.311+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 >> 172.21.15.31:0/2025631014 conn(0x7f3ad009c600 msgr2=0x7f3ad00cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.312+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 shutdown_connections 2024-01-19T04:56:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.312+0000 7f3ad858e640 1 -- 172.21.15.31:0/2025631014 wait complete. 2024-01-19T04:56:03.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:03.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.423+0000 7f4436089640 1 Processor -- start 2024-01-19T04:56:03.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.423+0000 7f4436089640 1 -- start start 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f4436089640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f443015d720 0x7f44300d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f4436089640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44300d3ea0 0x7f44300d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f4436089640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44300cf160 con 0x7f44300d1a50 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f4436089640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44300d1f00 con 0x7f443015d720 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f4436089640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44300ce8c0 con 0x7f44300d3ea0 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f442f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f443015d720 0x7f44300d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f442ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f44300d1a50 0x7f44300cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58410/0 (socket says 172.21.15.31:58410) 2024-01-19T04:56:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f442f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f443015d720 0x7f44300d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39896/0 (socket says 172.21.15.31:39896) 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.424+0000 7f442ffff640 1 -- 172.21.15.31:0/858729114 learned_addr learned my addr 172.21.15.31:0/858729114 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.425+0000 7f442effd640 1 --2- 172.21.15.31:0/858729114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44300d3ea0 0x7f44300d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.425+0000 7f442f7fe640 1 -- 172.21.15.31:0/858729114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44300d3ea0 msgr2=0x7f44300d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.425+0000 7f442f7fe640 1 --2- 172.21.15.31:0/858729114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44300d3ea0 0x7f44300d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.425+0000 7f442f7fe640 1 -- 172.21.15.31:0/858729114 >> v1:172.21.15.31:6789/0 conn(0x7f44300d1a50 legacy=0x7f44300cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.425+0000 7f442f7fe640 1 -- 172.21.15.31:0/858729114 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300ceba0 con 0x7f443015d720 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.425+0000 7f442e7fc640 1 -- 172.21.15.31:0/858729114 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755994154 0 0) 0x7f44300cf160 con 0x7f44300d1a50 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.425+0000 7f442f7fe640 1 --2- 172.21.15.31:0/858729114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443015d720 0x7f44300d0ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44200097a0 tx=0x7f442002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d010d599422eb0c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f442e7fc640 1 -- 172.21.15.31:0/858729114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442002f050 con 0x7f443015d720 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f442e7fc640 1 -- 172.21.15.31:0/858729114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420031700 con 0x7f443015d720 2024-01-19T04:56:03.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f442e7fc640 1 -- 172.21.15.31:0/858729114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420031a90 con 0x7f443015d720 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 -- 172.21.15.31:0/858729114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443015d720 msgr2=0x7f44300d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 --2- 172.21.15.31:0/858729114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443015d720 0x7f44300d0ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44200097a0 tx=0x7f442002b7a0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 -- 172.21.15.31:0/858729114 shutdown_connections 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 --2- 172.21.15.31:0/858729114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44300d3ea0 0x7f44300d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 --2- 172.21.15.31:0/858729114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443015d720 0x7f44300d0ee0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 -- 172.21.15.31:0/858729114 >> 172.21.15.31:0/858729114 conn(0x7f44300d3310 msgr2=0x7f44300d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 -- 172.21.15.31:0/858729114 shutdown_connections 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.426+0000 7f4436089640 1 -- 172.21.15.31:0/858729114 wait complete. 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.427+0000 7f4436089640 1 Processor -- start 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f4436089640 1 -- start start 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f4436089640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44300d1a50 0x7f44301f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f4436089640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44300d3ea0 0x7f443015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f4436089640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f443015d720 0x7f44300c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f4436089640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44301f7570 con 0x7f44300d3ea0 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f4436089640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443015e710 con 0x7f44300d1a50 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f4436089640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44301f6cd0 con 0x7f443015d720 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f442effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44300d3ea0 0x7f443015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f442effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44300d3ea0 0x7f443015a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39898/0 (socket says 172.21.15.31:39898) 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.428+0000 7f442effd640 1 -- 172.21.15.31:0/2901282149 learned_addr learned my addr 172.21.15.31:0/2901282149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f442effd640 1 -- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f443015d720 msgr2=0x7f44300c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f442effd640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f443015d720 0x7f44300c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f442effd640 1 -- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44300d1a50 msgr2=0x7f44301f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f442effd640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44300d1a50 0x7f44301f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f442effd640 1 -- 172.21.15.31:0/2901282149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420009400 con 0x7f44300d3ea0 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f442effd640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44300d3ea0 0x7f443015a770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f442400d7d0 tx=0x7f442400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f4417fff640 1 -- 172.21.15.31:0/2901282149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400b6c0 con 0x7f44300d3ea0 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f4417fff640 1 -- 172.21.15.31:0/2901282149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442400b840 con 0x7f44300d3ea0 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44301f6fb0 con 0x7f44300d3ea0 2024-01-19T04:56:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f4417fff640 1 -- 172.21.15.31:0/2901282149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400bbf0 con 0x7f44300d3ea0 2024-01-19T04:56:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.429+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44300cc770 con 0x7f44300d3ea0 2024-01-19T04:56:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.430+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4430074900 con 0x7f44300d3ea0 2024-01-19T04:56:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.430+0000 7f4417fff640 1 -- 172.21.15.31:0/2901282149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f442400bd50 con 0x7f44300d3ea0 2024-01-19T04:56:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.431+0000 7f4417fff640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f441c06af00 0x7f441c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.431+0000 7f4417fff640 1 -- 172.21.15.31:0/2901282149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4424089b60 con 0x7f44300d3ea0 2024-01-19T04:56:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.431+0000 7f442f7fe640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f441c06af00 0x7f441c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.432+0000 7f442f7fe640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f441c06af00 0x7f441c06d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4420035000 tx=0x7f4420030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.435+0000 7f4417fff640 1 -- 172.21.15.31:0/2901282149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4424054a40 con 0x7f44300d3ea0 2024-01-19T04:56:03.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.634+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f443015e880 con 0x7f44300d3ea0 2024-01-19T04:56:03.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.635+0000 7f4417fff640 1 -- 172.21.15.31:0/2901282149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4424058060 con 0x7f44300d3ea0 2024-01-19T04:56:03.634 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f441c06af00 msgr2=0x7f441c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f441c06af00 0x7f441c06d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4420035000 tx=0x7f4420030000 comp rx=0 tx=0).stop 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44300d3ea0 msgr2=0x7f443015a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44300d3ea0 0x7f443015a770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f442400d7d0 tx=0x7f442400dc90 comp rx=0 tx=0).stop 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 shutdown_connections 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f441c06af00 0x7f441c06d3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f443015d720 0x7f44300c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44300d3ea0 0x7f443015a770 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 --2- 172.21.15.31:0/2901282149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44300d1a50 0x7f44301f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 >> 172.21.15.31:0/2901282149 conn(0x7f44300d3310 msgr2=0x7f44300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 shutdown_connections 2024-01-19T04:56:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.637+0000 7f4436089640 1 -- 172.21.15.31:0/2901282149 wait complete. 2024-01-19T04:56:03.647 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:56:03.647 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:03.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.768+0000 7f7bcffec640 1 Processor -- start 2024-01-19T04:56:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.768+0000 7f7bcffec640 1 -- start start 2024-01-19T04:56:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcffec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc8144070 0x7f7bc81445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcffec640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7bc8144b30 0x7f7bc80b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcffec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bc80b2200 0x7f7bc806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcffec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc80b2690 con 0x7f7bc80b2200 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcffec640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc8144450 con 0x7f7bc8144b30 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcffec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80ced30 con 0x7f7bc8144070 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bc80b2200 0x7f7bc806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bc80b2200 0x7f7bc806a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39900/0 (socket says 172.21.15.31:39900) 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 -- 172.21.15.31:0/2294707978 learned_addr learned my addr 172.21.15.31:0/2294707978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcd587640 1 --2- 172.21.15.31:0/2294707978 >> v2:172.21.15.137:3300/0 conn(0x7f7bc8144b30 0x7f7bc80b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bcdd88640 1 --2- 172.21.15.31:0/2294707978 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc8144070 0x7f7bc81445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 -- 172.21.15.31:0/2294707978 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc8144070 msgr2=0x7f7bc81445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 --2- 172.21.15.31:0/2294707978 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc8144070 0x7f7bc81445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 -- 172.21.15.31:0/2294707978 >> v2:172.21.15.137:3300/0 conn(0x7f7bc8144b30 msgr2=0x7f7bc80b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 --2- 172.21.15.31:0/2294707978 >> v2:172.21.15.137:3300/0 conn(0x7f7bc8144b30 0x7f7bc80b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.769+0000 7f7bce589640 1 -- 172.21.15.31:0/2294707978 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80cf010 con 0x7f7bc80b2200 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bce589640 1 --2- 172.21.15.31:0/2294707978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc80b2200 0x7f7bc806a5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80b2df0 tx=0x7f7bc402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da420d32e3f2eb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bccd86640 1 -- 172.21.15.31:0/2294707978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc402ca70 con 0x7f7bc80b2200 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bccd86640 1 -- 172.21.15.31:0/2294707978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc402cbd0 con 0x7f7bc80b2200 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bccd86640 1 -- 172.21.15.31:0/2294707978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40343f0 con 0x7f7bc80b2200 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bcffec640 1 -- 172.21.15.31:0/2294707978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc80b2200 msgr2=0x7f7bc806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bcffec640 1 --2- 172.21.15.31:0/2294707978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc80b2200 0x7f7bc806a5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80b2df0 tx=0x7f7bc402bed0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bcffec640 1 -- 172.21.15.31:0/2294707978 shutdown_connections 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bcffec640 1 --2- 172.21.15.31:0/2294707978 >> v2:172.21.15.137:3300/0 conn(0x7f7bc8144b30 0x7f7bc80b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bcffec640 1 --2- 172.21.15.31:0/2294707978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc80b2200 0x7f7bc806a5b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.770+0000 7f7bcffec640 1 --2- 172.21.15.31:0/2294707978 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc8144070 0x7f7bc81445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.771+0000 7f7bcffec640 1 -- 172.21.15.31:0/2294707978 >> 172.21.15.31:0/2294707978 conn(0x7f7bc806d750 msgr2=0x7f7bc806db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.771+0000 7f7bcffec640 1 -- 172.21.15.31:0/2294707978 shutdown_connections 2024-01-19T04:56:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.771+0000 7f7bcffec640 1 -- 172.21.15.31:0/2294707978 wait complete. 2024-01-19T04:56:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.772+0000 7f7bcffec640 1 Processor -- start 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.772+0000 7f7bcffec640 1 -- start start 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc80b2200 0x7f7bc813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc8144070 0x7f7bc80a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8144b30 0x7f7bc80a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80a8db0 con 0x7f7bc8144b30 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80a9090 con 0x7f7bc8144070 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80a9370 con 0x7f7bc80b2200 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8144b30 0x7f7bc80a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8144b30 0x7f7bc80a9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39902/0 (socket says 172.21.15.31:39902) 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 -- 172.21.15.31:0/3851269803 learned_addr learned my addr 172.21.15.31:0/3851269803 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcd587640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc8144070 0x7f7bc80a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcdd88640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc80b2200 0x7f7bc813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 -- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc80b2200 msgr2=0x7f7bc813d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc80b2200 0x7f7bc813d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 -- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc8144070 msgr2=0x7f7bc80a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc8144070 0x7f7bc80a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 -- 172.21.15.31:0/3851269803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc4009b30 con 0x7f7bc8144b30 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bce589640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8144b30 0x7f7bc80a9990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009e50 tx=0x7f7bc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bbe7fc640 1 -- 172.21.15.31:0/3851269803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4009510 con 0x7f7bc8144b30 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bbe7fc640 1 -- 172.21.15.31:0/3851269803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc403e450 con 0x7f7bc8144b30 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc815b940 con 0x7f7bc8144b30 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bbe7fc640 1 -- 172.21.15.31:0/3851269803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc403e800 con 0x7f7bc8144b30 2024-01-19T04:56:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.773+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc815e4e0 con 0x7f7bc8144b30 2024-01-19T04:56:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.774+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc80696e0 con 0x7f7bc8144b30 2024-01-19T04:56:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.775+0000 7f7bbe7fc640 1 -- 172.21.15.31:0/3851269803 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7bc403e960 con 0x7f7bc8144b30 2024-01-19T04:56:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.775+0000 7f7bbe7fc640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7bac06aed0 0x7f7bac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.775+0000 7f7bbe7fc640 1 -- 172.21.15.31:0/3851269803 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7bc40ac300 con 0x7f7bc8144b30 2024-01-19T04:56:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.776+0000 7f7bcdd88640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7bac06aed0 0x7f7bac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.776+0000 7f7bcdd88640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7bac06aed0 0x7f7bac06d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0006fd0 tx=0x7f7bc0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.780+0000 7f7bbe7fc640 1 -- 172.21.15.31:0/3851269803 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7bc4077200 con 0x7f7bc8144b30 2024-01-19T04:56:03.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.960+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bc815bb30 con 0x7f7bc8144b30 2024-01-19T04:56:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.962+0000 7f7bbe7fc640 1 -- 172.21.15.31:0/3851269803 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16720 (secure 0 0 0) 0x7f7bc407a820 con 0x7f7bc8144b30 2024-01-19T04:56:03.961 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:03.961 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:01.578429+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7bac06aed0 msgr2=0x7f7bac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7bac06aed0 0x7f7bac06d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0006fd0 tx=0x7f7bc0008040 comp rx=0 tx=0).stop 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8144b30 msgr2=0x7f7bc80a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8144b30 0x7f7bc80a9990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009e50 tx=0x7f7bc4004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 shutdown_connections 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7bac06aed0 0x7f7bac06d390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8144b30 0x7f7bc80a9990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc8144070 0x7f7bc80a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 --2- 172.21.15.31:0/3851269803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7bc80b2200 0x7f7bc813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 >> 172.21.15.31:0/3851269803 conn(0x7f7bc806d750 msgr2=0x7f7bc81425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 shutdown_connections 2024-01-19T04:56:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:03.964+0000 7f7bcffec640 1 -- 172.21.15.31:0/3851269803 wait complete. 2024-01-19T04:56:03.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:03.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:04.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.079+0000 7f3df011d640 1 Processor -- start 2024-01-19T04:56:04.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.079+0000 7f3df011d640 1 -- start start 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.079+0000 7f3df011d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 0x7f3de815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3df011d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de815dee0 con 0x7f3de80d0ef0 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3df011d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de815e1c0 con 0x7f3de80d3e50 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3df011d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815e4a0 con 0x7f3de80d18e0 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3dee6ba640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3de80d0ef0 0x7f3de80cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58426/0 (socket says 172.21.15.31:58426) 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3dee6ba640 1 -- 172.21.15.31:0/529870257 learned_addr learned my addr 172.21.15.31:0/529870257 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 0x7f3de815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155297412 0 0) 0x7f3de815dee0 con 0x7f3de80d0ef0 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd4003520 con 0x7f3de80d0ef0 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 403859179 0 0) 0x7f3dd4003520 con 0x7f3de80d0ef0 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 msgr2=0x7f3de815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 0x7f3de815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 >> v1:172.21.15.137:6789/0 conn(0x7f3de80d3e50 legacy=0x7f3de80d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de815e780 con 0x7f3de80d0ef0 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f3de40019f0 con 0x7f3de80d0ef0 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 >> v1:172.21.15.31:6789/0 conn(0x7f3de80d0ef0 legacy=0x7f3de80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40039a0 0x7f3dd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 0x7f3dd40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dd4004d00 0x7f3dd4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815e4a0 con 0x7f3dd40039a0 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd400ba60 con 0x7f3dd4004d00 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd400bc20 con 0x7f3de80d18e0 2024-01-19T04:56:04.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.080+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40039a0 0x7f3dd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3ded6b8640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 0x7f3dd40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3dee6ba640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dd4004d00 0x7f3dd4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3dedeb9640 1 -- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 msgr2=0x7f3dd40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3de80d18e0 0x7f3dd40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3dedeb9640 1 -- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dd4004d00 msgr2=0x7f3dd4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dd4004d00 0x7f3dd4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3dedeb9640 1 -- 172.21.15.31:0/529870257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4003520 con 0x7f3dd40039a0 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.081+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40039a0 0x7f3dd4003dc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009a90 tx=0x7f3dd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d3c4bc36e61cb7 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de002c680 con 0x7f3dd40039a0 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de002c7e0 con 0x7f3dd40039a0 2024-01-19T04:56:04.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3deceb7640 1 -- 172.21.15.31:0/529870257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de002cb70 con 0x7f3dd40039a0 2024-01-19T04:56:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3df011d640 1 -- 172.21.15.31:0/529870257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40039a0 msgr2=0x7f3dd4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3df011d640 1 --2- 172.21.15.31:0/529870257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40039a0 0x7f3dd4003dc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009a90 tx=0x7f3dd40057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3dee6ba640 1 -- 172.21.15.31:0/529870257 reap_dead start 2024-01-19T04:56:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3df011d640 1 -- 172.21.15.31:0/529870257 shutdown_connections 2024-01-19T04:56:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3df011d640 1 -- 172.21.15.31:0/529870257 >> 172.21.15.31:0/529870257 conn(0x7f3de809c600 msgr2=0x7f3de80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:04.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3df011d640 1 -- 172.21.15.31:0/529870257 shutdown_connections 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.082+0000 7f3df011d640 1 -- 172.21.15.31:0/529870257 wait complete. 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.083+0000 7f3df011d640 1 Processor -- start 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.083+0000 7f3df011d640 1 -- start start 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3df011d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3dd40039a0 0x7f3de80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3df011d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de80cc670 0x7f3de81f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3df011d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de81f3610 0x7f3de80c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3df011d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de81f4070 con 0x7f3de81f3610 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3df011d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de81f3ab0 con 0x7f3de80cc670 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3df011d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de81f3d90 con 0x7f3dd40039a0 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3dee6ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de81f3610 0x7f3de80c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3dee6ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de81f3610 0x7f3de80c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39918/0 (socket says 172.21.15.31:39918) 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3dee6ba640 1 -- 172.21.15.31:0/3035661349 learned_addr learned my addr 172.21.15.31:0/3035661349 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3dd40039a0 0x7f3de80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.084+0000 7f3ded6b8640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de80cc670 0x7f3de81f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dee6ba640 1 -- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3dd40039a0 msgr2=0x7f3de80b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dee6ba640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3dd40039a0 0x7f3de80b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dee6ba640 1 -- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de80cc670 msgr2=0x7f3de81f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dee6ba640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de80cc670 0x7f3de81f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dee6ba640 1 -- 172.21.15.31:0/3035661349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de80c6f80 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dee6ba640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de81f3610 0x7f3de80c7780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3de80a1240 tx=0x7f3de400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dde7fc640 1 -- 172.21.15.31:0/3035661349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400ec10 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dde7fc640 1 -- 172.21.15.31:0/3035661349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de400ed70 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de80c7260 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3dde7fc640 1 -- 172.21.15.31:0/3035661349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400eee0 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.085+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de80c7cc0 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.086+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de815dee0 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.086+0000 7f3dde7fc640 1 -- 172.21.15.31:0/3035661349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3de4013660 con 0x7f3de81f3610 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.087+0000 7f3dde7fc640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3dcc06aed0 0x7f3dcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.087+0000 7f3dde7fc640 1 -- 172.21.15.31:0/3035661349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3de40899d0 con 0x7f3de81f3610 2024-01-19T04:56:04.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.087+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3dcc06aed0 0x7f3dcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.088+0000 7f3dedeb9640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3dcc06aed0 0x7f3dcc06d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3de0036000 tx=0x7f3de0030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.091+0000 7f3dde7fc640 1 -- 172.21.15.31:0/3035661349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3de4054960 con 0x7f3de81f3610 2024-01-19T04:56:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.268+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3de8074900 con 0x7f3de81f3610 2024-01-19T04:56:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.268+0000 7f3dde7fc640 1 -- 172.21.15.31:0/3035661349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3de4057f80 con 0x7f3de81f3610 2024-01-19T04:56:04.268 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3dcc06aed0 msgr2=0x7f3dcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3dcc06aed0 0x7f3dcc06d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3de0036000 tx=0x7f3de0030000 comp rx=0 tx=0).stop 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de81f3610 msgr2=0x7f3de80c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de81f3610 0x7f3de80c7780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3de80a1240 tx=0x7f3de400e530 comp rx=0 tx=0).stop 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 shutdown_connections 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3dcc06aed0 0x7f3dcc06d390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de81f3610 0x7f3de80c7780 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de80cc670 0x7f3de81f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 --2- 172.21.15.31:0/3035661349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3dd40039a0 0x7f3de80b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 >> 172.21.15.31:0/3035661349 conn(0x7f3de809c600 msgr2=0x7f3de80b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 shutdown_connections 2024-01-19T04:56:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.270+0000 7f3df011d640 1 -- 172.21.15.31:0/3035661349 wait complete. 2024-01-19T04:56:04.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:04.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.385+0000 7fdd56607640 1 Processor -- start 2024-01-19T04:56:04.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.385+0000 7fdd56607640 1 -- start start 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.385+0000 7fdd56607640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd5015d3a0 0x7fdd5006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.385+0000 7fdd56607640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fdd500cf570 0x7fdd50161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.385+0000 7fdd56607640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd500cf9f0 con 0x7fdd5015d780 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.385+0000 7fdd56607640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd500cfcd0 con 0x7fdd500cf570 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.385+0000 7fdd56607640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500cffb0 con 0x7fdd5015d3a0 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4ffff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd5015d3a0 0x7fdd5006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4f7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fdd500cf570 0x7fdd50161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd54ba4640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fdd5015d780 0x7fdd500cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:59942/0 (socket says 172.21.15.31:59942) 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd54ba4640 1 -- 172.21.15.31:0/3157764252 learned_addr learned my addr 172.21.15.31:0/3157764252 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4effd640 1 -- 172.21.15.31:0/3157764252 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572321502 0 0) 0x7fdd500cf9f0 con 0x7fdd5015d780 2024-01-19T04:56:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4effd640 1 -- 172.21.15.31:0/3157764252 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd38003520 con 0x7fdd5015d780 2024-01-19T04:56:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4ffff640 1 -- 172.21.15.31:0/3157764252 >> v2:172.21.15.137:3300/0 conn(0x7fdd500cf570 msgr2=0x7fdd50161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4ffff640 1 --2- 172.21.15.31:0/3157764252 >> v2:172.21.15.137:3300/0 conn(0x7fdd500cf570 0x7fdd50161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4ffff640 1 -- 172.21.15.31:0/3157764252 >> v1:172.21.15.137:6789/0 conn(0x7fdd5015d780 legacy=0x7fdd500cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4ffff640 1 -- 172.21.15.31:0/3157764252 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd500d0290 con 0x7fdd5015d3a0 2024-01-19T04:56:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.386+0000 7fdd4f7fe640 1 --2- 172.21.15.31:0/3157764252 >> v2:172.21.15.137:3300/0 conn(0x7fdd500cf570 0x7fdd50161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.387+0000 7fdd4ffff640 1 --2- 172.21.15.31:0/3157764252 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd5015d3a0 0x7fdd5006c4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009790 tx=0x7fdd4002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb0596766bb372b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.387+0000 7fdd4effd640 1 -- 172.21.15.31:0/3157764252 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40031800 con 0x7fdd5015d3a0 2024-01-19T04:56:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.387+0000 7fdd4effd640 1 -- 172.21.15.31:0/3157764252 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40031960 con 0x7fdd5015d3a0 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.387+0000 7fdd4effd640 1 -- 172.21.15.31:0/3157764252 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40031cd0 con 0x7fdd5015d3a0 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 -- 172.21.15.31:0/3157764252 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd5015d3a0 msgr2=0x7fdd5006c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 --2- 172.21.15.31:0/3157764252 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd5015d3a0 0x7fdd5006c4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009790 tx=0x7fdd4002b8a0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 -- 172.21.15.31:0/3157764252 shutdown_connections 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 --2- 172.21.15.31:0/3157764252 >> v2:172.21.15.137:3300/0 conn(0x7fdd500cf570 0x7fdd50161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 --2- 172.21.15.31:0/3157764252 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd5015d3a0 0x7fdd5006c4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 -- 172.21.15.31:0/3157764252 >> 172.21.15.31:0/3157764252 conn(0x7fdd500d1100 msgr2=0x7fdd500d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 -- 172.21.15.31:0/3157764252 shutdown_connections 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.388+0000 7fdd56607640 1 -- 172.21.15.31:0/3157764252 wait complete. 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.389+0000 7fdd56607640 1 Processor -- start 2024-01-19T04:56:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.389+0000 7fdd56607640 1 -- start start 2024-01-19T04:56:04.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd56607640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd500cf570 0x7fdd500c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd56607640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd5015d3a0 0x7fdd500c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd56607640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd5015d780 0x7fdd5009e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd56607640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500c5540 con 0x7fdd5015d780 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd56607640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500c5820 con 0x7fdd5015d3a0 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd56607640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500c5b00 con 0x7fdd500cf570 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd5015d780 0x7fdd5009e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd5015d780 0x7fdd5009e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39928/0 (socket says 172.21.15.31:39928) 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 -- 172.21.15.31:0/3156679829 learned_addr learned my addr 172.21.15.31:0/3156679829 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd4ffff640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd500cf570 0x7fdd500c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd4f7fe640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd5015d3a0 0x7fdd500c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 -- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd500cf570 msgr2=0x7fdd500c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd500cf570 0x7fdd500c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 -- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd5015d3a0 msgr2=0x7fdd500c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd5015d3a0 0x7fdd500c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.390+0000 7fdd54ba4640 1 -- 172.21.15.31:0/3156679829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44009400 con 0x7fdd5015d780 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.391+0000 7fdd54ba4640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd5015d780 0x7fdd5009e260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800caa0 tx=0x7fdd4800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.391+0000 7fdd4cff9640 1 -- 172.21.15.31:0/3156679829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800ac00 con 0x7fdd5015d780 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.391+0000 7fdd4cff9640 1 -- 172.21.15.31:0/3156679829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800ad60 con 0x7fdd5015d780 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.391+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd400093a0 con 0x7fdd5015d780 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.391+0000 7fdd4cff9640 1 -- 172.21.15.31:0/3156679829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48012620 con 0x7fdd5015d780 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.391+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd500c60c0 con 0x7fdd5015d780 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.392+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd500c62b0 con 0x7fdd5015d780 2024-01-19T04:56:04.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.392+0000 7fdd4cff9640 1 -- 172.21.15.31:0/3156679829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdd48007070 con 0x7fdd5015d780 2024-01-19T04:56:04.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.393+0000 7fdd4cff9640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd3c06aed0 0x7fdd3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.393+0000 7fdd4cff9640 1 -- 172.21.15.31:0/3156679829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdd48089690 con 0x7fdd5015d780 2024-01-19T04:56:04.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.393+0000 7fdd4ffff640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd3c06aed0 0x7fdd3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.393+0000 7fdd4ffff640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd3c06aed0 0x7fdd3c06d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd40036000 tx=0x7fdd40008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.397+0000 7fdd4cff9640 1 -- 172.21.15.31:0/3156679829 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdd480544f0 con 0x7fdd5015d780 2024-01-19T04:56:04.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.576+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd50074900 con 0x7fdd5015d780 2024-01-19T04:56:04.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.576+0000 7fdd4cff9640 1 -- 172.21.15.31:0/3156679829 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd48057b10 con 0x7fdd5015d780 2024-01-19T04:56:04.576 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd3c06aed0 msgr2=0x7fdd3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd3c06aed0 0x7fdd3c06d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd40036000 tx=0x7fdd40008030 comp rx=0 tx=0).stop 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd5015d780 msgr2=0x7fdd5009e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd5015d780 0x7fdd5009e260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800caa0 tx=0x7fdd4800cf60 comp rx=0 tx=0).stop 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 shutdown_connections 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd3c06aed0 0x7fdd3c06d390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd5015d780 0x7fdd5009e260 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd5015d3a0 0x7fdd500c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 --2- 172.21.15.31:0/3156679829 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd500cf570 0x7fdd500c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 >> 172.21.15.31:0/3156679829 conn(0x7fdd500d1100 msgr2=0x7fdd5006f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:04.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 shutdown_connections 2024-01-19T04:56:04.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.578+0000 7fdd56607640 1 -- 172.21.15.31:0/3156679829 wait complete. 2024-01-19T04:56:04.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.692+0000 7fe2d5502640 1 Processor -- start 2024-01-19T04:56:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.692+0000 7fe2d5502640 1 -- start start 2024-01-19T04:56:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2d5502640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2d015b8c0 0x7fe2d015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2d5502640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe2d0072110 0x7fe2d013ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2d5502640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d015f280 con 0x7fe2d013e8e0 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2d5502640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d015fd50 con 0x7fe2d015b8c0 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2d5502640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d0071570 con 0x7fe2d0072110 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ceffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2d015b8c0 0x7fe2d015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ceffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2d015b8c0 0x7fe2d015ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39938/0 (socket says 172.21.15.31:39938) 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ce7fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe2d013e8e0 0x7fe2d0071a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58438/0 (socket says 172.21.15.31:58438) 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ceffd640 1 -- 172.21.15.31:0/1142559647 learned_addr learned my addr 172.21.15.31:0/1142559647 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2cf7fe640 1 --2- 172.21.15.31:0/1142559647 >> v2:172.21.15.137:3300/0 conn(0x7fe2d0072110 0x7fe2d013ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ceffd640 1 -- 172.21.15.31:0/1142559647 >> v2:172.21.15.137:3300/0 conn(0x7fe2d0072110 msgr2=0x7fe2d013ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ceffd640 1 --2- 172.21.15.31:0/1142559647 >> v2:172.21.15.137:3300/0 conn(0x7fe2d0072110 0x7fe2d013ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2cdffb640 1 -- 172.21.15.31:0/1142559647 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189587257 0 0) 0x7fe2d015f280 con 0x7fe2d013e8e0 2024-01-19T04:56:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ceffd640 1 -- 172.21.15.31:0/1142559647 >> v1:172.21.15.31:6789/0 conn(0x7fe2d013e8e0 legacy=0x7fe2d0071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2ceffd640 1 -- 172.21.15.31:0/1142559647 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d015f4b0 con 0x7fe2d015b8c0 2024-01-19T04:56:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.693+0000 7fe2cf7fe640 1 --2- 172.21.15.31:0/1142559647 >> v2:172.21.15.137:3300/0 conn(0x7fe2d0072110 0x7fe2d013ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.694+0000 7fe2ceffd640 1 --2- 172.21.15.31:0/1142559647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d015b8c0 0x7fe2d015ed40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00afb70 tx=0x7fe2c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63cd4506e3dec9a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.694+0000 7fe2cdffb640 1 -- 172.21.15.31:0/1142559647 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8030860 con 0x7fe2d015b8c0 2024-01-19T04:56:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.694+0000 7fe2cdffb640 1 -- 172.21.15.31:0/1142559647 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c80309c0 con 0x7fe2d015b8c0 2024-01-19T04:56:04.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.694+0000 7fe2cdffb640 1 -- 172.21.15.31:0/1142559647 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8030d50 con 0x7fe2d015b8c0 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 -- 172.21.15.31:0/1142559647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d015b8c0 msgr2=0x7fe2d015ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1142559647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d015b8c0 0x7fe2d015ed40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00afb70 tx=0x7fe2c802b9a0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 -- 172.21.15.31:0/1142559647 shutdown_connections 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1142559647 >> v2:172.21.15.137:3300/0 conn(0x7fe2d0072110 0x7fe2d013ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1142559647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d015b8c0 0x7fe2d015ed40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 -- 172.21.15.31:0/1142559647 >> 172.21.15.31:0/1142559647 conn(0x7fe2d00b1c50 msgr2=0x7fe2d00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 -- 172.21.15.31:0/1142559647 shutdown_connections 2024-01-19T04:56:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.695+0000 7fe2d5502640 1 -- 172.21.15.31:0/1142559647 wait complete. 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.696+0000 7fe2d5502640 1 Processor -- start 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.696+0000 7fe2d5502640 1 -- start start 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2d5502640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0072110 0x7fe2d00cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2d5502640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d013e8e0 0x7fe2d00ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2d5502640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2d015b8c0 0x7fe2d00c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2d5502640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00cfa70 con 0x7fe2d013e8e0 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2d5502640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c9310 con 0x7fe2d0072110 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2d5502640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c95f0 con 0x7fe2d015b8c0 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d013e8e0 0x7fe2d00ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d013e8e0 0x7fe2d00ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39950/0 (socket says 172.21.15.31:39950) 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 -- 172.21.15.31:0/1782830349 learned_addr learned my addr 172.21.15.31:0/1782830349 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ceffd640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0072110 0x7fe2d00cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 -- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2d015b8c0 msgr2=0x7fe2d00c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2cf7fe640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2d015b8c0 0x7fe2d00c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2d015b8c0 0x7fe2d00c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 -- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0072110 msgr2=0x7fe2d00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0072110 0x7fe2d00cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 -- 172.21.15.31:0/1782830349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80094d0 con 0x7fe2d013e8e0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ceffd640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0072110 0x7fe2d00cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2cf7fe640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2d015b8c0 0x7fe2d00c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.697+0000 7fe2ce7fc640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d013e8e0 0x7fe2d00ace10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800c4b0 tx=0x7fe2b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.698+0000 7fe2c77fe640 1 -- 172.21.15.31:0/1782830349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8007480 con 0x7fe2d013e8e0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.698+0000 7fe2c77fe640 1 -- 172.21.15.31:0/1782830349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b80075e0 con 0x7fe2d013e8e0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.698+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d00c98d0 con 0x7fe2d013e8e0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.698+0000 7fe2c77fe640 1 -- 172.21.15.31:0/1782830349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8004a20 con 0x7fe2d013e8e0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.698+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d00c9e90 con 0x7fe2d013e8e0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.699+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d00ca080 con 0x7fe2d013e8e0 2024-01-19T04:56:04.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.699+0000 7fe2c77fe640 1 -- 172.21.15.31:0/1782830349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe2b8004b80 con 0x7fe2d013e8e0 2024-01-19T04:56:04.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.699+0000 7fe2c77fe640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2b006aed0 0x7fe2b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:04.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.699+0000 7fe2c77fe640 1 -- 172.21.15.31:0/1782830349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe2b8088c20 con 0x7fe2d013e8e0 2024-01-19T04:56:04.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.700+0000 7fe2ceffd640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2b006aed0 0x7fe2b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:04.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.700+0000 7fe2ceffd640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2b006aed0 0x7fe2b006d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8036000 tx=0x7fe2c803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:04.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.704+0000 7fe2c77fe640 1 -- 172.21.15.31:0/1782830349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe2b8053a90 con 0x7fe2d013e8e0 2024-01-19T04:56:04.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.885+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2d0074900 con 0x7fe2d013e8e0 2024-01-19T04:56:04.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.886+0000 7fe2c77fe640 1 -- 172.21.15.31:0/1782830349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2b80180a0 con 0x7fe2d013e8e0 2024-01-19T04:56:04.885 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2b006aed0 msgr2=0x7fe2b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2b006aed0 0x7fe2b006d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8036000 tx=0x7fe2c803b040 comp rx=0 tx=0).stop 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d013e8e0 msgr2=0x7fe2d00ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d013e8e0 0x7fe2d00ace10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800c4b0 tx=0x7fe2b800c970 comp rx=0 tx=0).stop 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 shutdown_connections 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2b006aed0 0x7fe2b006d390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2d015b8c0 0x7fe2d00c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d013e8e0 0x7fe2d00ace10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 --2- 172.21.15.31:0/1782830349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0072110 0x7fe2d00cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 >> 172.21.15.31:0/1782830349 conn(0x7fe2d00b1c50 msgr2=0x7fe2d00c4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 shutdown_connections 2024-01-19T04:56:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:04.888+0000 7fe2d5502640 1 -- 172.21.15.31:0/1782830349 wait complete. 2024-01-19T04:56:04.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.000+0000 7fdcbf256640 1 Processor -- start 2024-01-19T04:56:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbf256640 1 -- start start 2024-01-19T04:56:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbf256640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdcb80d1fe0 0x7fdcb80d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbf256640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80cef10 con 0x7fdcb815d700 2024-01-19T04:56:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbf256640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80ce300 con 0x7fdcb80d11b0 2024-01-19T04:56:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbf256640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb80ce5e0 con 0x7fdcb80d1fe0 2024-01-19T04:56:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbcff2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdcb815d700 0x7fdcb80d18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58454/0 (socket says 172.21.15.31:58454) 2024-01-19T04:56:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbcff2640 1 -- 172.21.15.31:0/2107904725 learned_addr learned my addr 172.21.15.31:0/2107904725 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.001+0000 7fdcbd7f3640 1 --2- 172.21.15.31:0/2107904725 >> v2:172.21.15.31:3300/0 conn(0x7fdcb80d1fe0 0x7fdcb80d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.002+0000 7fdcbd7f3640 1 -- 172.21.15.31:0/2107904725 >> v1:172.21.15.137:6789/0 conn(0x7fdcb80d11b0 legacy=0x7fdcb80b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.002+0000 7fdcbd7f3640 1 -- 172.21.15.31:0/2107904725 >> v1:172.21.15.31:6789/0 conn(0x7fdcb815d700 legacy=0x7fdcb80d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.002+0000 7fdcb77fe640 1 -- 172.21.15.31:0/2107904725 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573033138 0 0) 0x7fdcb80cef10 con 0x7fdcb815d700 2024-01-19T04:56:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.002+0000 7fdcbd7f3640 1 -- 172.21.15.31:0/2107904725 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80ce8c0 con 0x7fdcb80d1fe0 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.002+0000 7fdcbd7f3640 1 --2- 172.21.15.31:0/2107904725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb80d1fe0 0x7fdcb80d3e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ad07f7031964fd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcb77fe640 1 -- 172.21.15.31:0/2107904725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb002cd00 con 0x7fdcb80d1fe0 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcb77fe640 1 -- 172.21.15.31:0/2107904725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb002ce60 con 0x7fdcb80d1fe0 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcb77fe640 1 -- 172.21.15.31:0/2107904725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0034660 con 0x7fdcb80d1fe0 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcbf256640 1 -- 172.21.15.31:0/2107904725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb80d1fe0 msgr2=0x7fdcb80d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2107904725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb80d1fe0 0x7fdcb80d3e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb002bf50 comp rx=0 tx=0).stop 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcbf256640 1 -- 172.21.15.31:0/2107904725 shutdown_connections 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2107904725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb80d1fe0 0x7fdcb80d3e10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcbf256640 1 -- 172.21.15.31:0/2107904725 >> 172.21.15.31:0/2107904725 conn(0x7fdcb80d32f0 msgr2=0x7fdcb80d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcbf256640 1 -- 172.21.15.31:0/2107904725 shutdown_connections 2024-01-19T04:56:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.003+0000 7fdcbf256640 1 -- 172.21.15.31:0/2107904725 wait complete. 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.004+0000 7fdcbf256640 1 Processor -- start 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.004+0000 7fdcbf256640 1 -- start start 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbf256640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb80d11b0 0x7fdcb814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbf256640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdcb80d1fe0 0x7fdcb80c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbf256640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb815d700 0x7fdcb81593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbf256640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80cab20 con 0x7fdcb815d700 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbf256640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c71e0 con 0x7fdcb80d11b0 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbf256640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb814cb30 con 0x7fdcb80d1fe0 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbd7f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb815d700 0x7fdcb81593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbd7f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb815d700 0x7fdcb81593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39968/0 (socket says 172.21.15.31:39968) 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.005+0000 7fdcbd7f3640 1 -- 172.21.15.31:0/2912902612 learned_addr learned my addr 172.21.15.31:0/2912902612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbd7f3640 1 -- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdcb80d1fe0 msgr2=0x7fdcb80c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbcff2640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb80d11b0 0x7fdcb814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcb7fff640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdcb80d1fe0 0x7fdcb80c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbd7f3640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdcb80d1fe0 0x7fdcb80c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbd7f3640 1 -- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb80d11b0 msgr2=0x7fdcb814c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbd7f3640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb80d11b0 0x7fdcb814c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbd7f3640 1 -- 172.21.15.31:0/2912902612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0009a70 con 0x7fdcb815d700 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbd7f3640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb815d700 0x7fdcb81593c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcb57fa640 1 -- 172.21.15.31:0/2912902612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb003e670 con 0x7fdcb815d700 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcb57fa640 1 -- 172.21.15.31:0/2912902612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb003e7f0 con 0x7fdcb815d700 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb80c9900 con 0x7fdcb815d700 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcb57fa640 1 -- 172.21.15.31:0/2912902612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb003eba0 con 0x7fdcb815d700 2024-01-19T04:56:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.006+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb80ca740 con 0x7fdcb815d700 2024-01-19T04:56:05.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.007+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80cac60 con 0x7fdcb815d700 2024-01-19T04:56:05.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.008+0000 7fdcb57fa640 1 -- 172.21.15.31:0/2912902612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdcb003b040 con 0x7fdcb815d700 2024-01-19T04:56:05.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.008+0000 7fdcb57fa640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdc9c06aed0 0x7fdc9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.008+0000 7fdcb57fa640 1 -- 172.21.15.31:0/2912902612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdcb00ac1d0 con 0x7fdcb815d700 2024-01-19T04:56:05.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.008+0000 7fdcbcff2640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdc9c06aed0 0x7fdc9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.009+0000 7fdcbcff2640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdc9c06aed0 0x7fdc9c06d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcac000f60 tx=0x7fdcac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.012+0000 7fdcb57fa640 1 -- 172.21.15.31:0/2912902612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdcb80cac60 con 0x7fdcb815d700 2024-01-19T04:56:05.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.203+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdcb80cac60 con 0x7fdcb815d700 2024-01-19T04:56:05.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.203+0000 7fdcb57fa640 1 -- 172.21.15.31:0/2912902612 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdcb80cac60 con 0x7fdcb815d700 2024-01-19T04:56:05.203 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdc9c06aed0 msgr2=0x7fdc9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdc9c06aed0 0x7fdc9c06d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcac000f60 tx=0x7fdcac009040 comp rx=0 tx=0).stop 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb815d700 msgr2=0x7fdcb81593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb815d700 0x7fdcb81593c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb0004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 shutdown_connections 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdc9c06aed0 0x7fdc9c06d390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcb815d700 0x7fdcb81593c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdcb80d1fe0 0x7fdcb80c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 --2- 172.21.15.31:0/2912902612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb80d11b0 0x7fdcb814c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 >> 172.21.15.31:0/2912902612 conn(0x7fdcb80d32f0 msgr2=0x7fdcb815b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:05.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 shutdown_connections 2024-01-19T04:56:05.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.205+0000 7fdcbf256640 1 -- 172.21.15.31:0/2912902612 wait complete. 2024-01-19T04:56:05.215 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi137.front.sepia.ceph.com... 2024-01-19T04:56:05.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:05.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-01-19T04:56:05.249 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:05.249 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:56:05.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:05.249 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-19T04:56:05.306 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-19T04:56:05.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:05.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.408+0000 7f4db8f68640 1 Processor -- start 2024-01-19T04:56:05.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.408+0000 7f4db8f68640 1 -- start start 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.408+0000 7f4db8f68640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4db40ca310 0x7f4db406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.408+0000 7f4db8f68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4db406b6d0 0x7f4db4164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.408+0000 7f4db8f68640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db406ef80 con 0x7f4db40cba90 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.408+0000 7f4db8f68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db406db30 con 0x7f4db406b6d0 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.408+0000 7f4db8f68640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db406cf30 con 0x7f4db40ca310 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db2d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4db406b6d0 0x7f4db4164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db2d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4db406b6d0 0x7f4db4164ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39978/0 (socket says 172.21.15.31:39978) 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db2d9d640 1 -- 172.21.15.31:0/3508684490 learned_addr learned my addr 172.21.15.31:0/3508684490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db259c640 1 --2- 172.21.15.31:0/3508684490 >> v2:172.21.15.137:3300/0 conn(0x7f4db40ca310 0x7f4db406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db2d9d640 1 -- 172.21.15.31:0/3508684490 >> v2:172.21.15.137:3300/0 conn(0x7f4db40ca310 msgr2=0x7f4db406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db2d9d640 1 --2- 172.21.15.31:0/3508684490 >> v2:172.21.15.137:3300/0 conn(0x7f4db40ca310 0x7f4db406d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db2d9d640 1 -- 172.21.15.31:0/3508684490 >> v1:172.21.15.137:6789/0 conn(0x7f4db40cba90 legacy=0x7f4db406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.409+0000 7f4db2d9d640 1 -- 172.21.15.31:0/3508684490 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db406f1b0 con 0x7f4db406b6d0 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db2d9d640 1 --2- 172.21.15.31:0/3508684490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db406b6d0 0x7f4db4164ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4db40ccd40 tx=0x7f4dac02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25223d854198e4f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db1d9b640 1 -- 172.21.15.31:0/3508684490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac02c9f0 con 0x7f4db406b6d0 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db1d9b640 1 -- 172.21.15.31:0/3508684490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac02cb50 con 0x7f4db406b6d0 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db1d9b640 1 -- 172.21.15.31:0/3508684490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0343f0 con 0x7f4db406b6d0 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 -- 172.21.15.31:0/3508684490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db406b6d0 msgr2=0x7f4db4164ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 --2- 172.21.15.31:0/3508684490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db406b6d0 0x7f4db4164ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4db40ccd40 tx=0x7f4dac02be50 comp rx=0 tx=0).stop 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 -- 172.21.15.31:0/3508684490 shutdown_connections 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 --2- 172.21.15.31:0/3508684490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db406b6d0 0x7f4db4164ea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 --2- 172.21.15.31:0/3508684490 >> v2:172.21.15.137:3300/0 conn(0x7f4db40ca310 0x7f4db406d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 -- 172.21.15.31:0/3508684490 >> 172.21.15.31:0/3508684490 conn(0x7f4db40cc1b0 msgr2=0x7f4db40cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 -- 172.21.15.31:0/3508684490 shutdown_connections 2024-01-19T04:56:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.410+0000 7f4db8f68640 1 -- 172.21.15.31:0/3508684490 wait complete. 2024-01-19T04:56:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 Processor -- start 2024-01-19T04:56:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 -- start start 2024-01-19T04:56:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db406b6d0 0x7f4db4150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db40ca310 0x7f4db40ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40cba90 0x7f4db414dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db414fba0 con 0x7f4db40ca310 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41509e0 con 0x7f4db406b6d0 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4db8f68640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db414fdd0 con 0x7f4db40cba90 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.412+0000 7f4dabfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db40ca310 0x7f4db40ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db40ca310 0x7f4db40ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39986/0 (socket says 172.21.15.31:39986) 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 -- 172.21.15.31:0/457598716 learned_addr learned my addr 172.21.15.31:0/457598716 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4db259c640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db406b6d0 0x7f4db4150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4db2d9d640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40cba90 0x7f4db414dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 -- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40cba90 msgr2=0x7f4db414dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40cba90 0x7f4db414dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 -- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db406b6d0 msgr2=0x7f4db4150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db406b6d0 0x7f4db4150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 -- 172.21.15.31:0/457598716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac009b30 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4db2d9d640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40cba90 0x7f4db414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4db259c640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db406b6d0 0x7f4db4150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4dabfff640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db40ca310 0x7f4db40ca960 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00b3e0 tx=0x7f4d9c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4daaffd640 1 -- 172.21.15.31:0/457598716 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c011830 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4daaffd640 1 -- 172.21.15.31:0/457598716 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c0119b0 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db41500b0 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4daaffd640 1 -- 172.21.15.31:0/457598716 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c011d60 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.413+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db4150670 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.414+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db40696e0 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.415+0000 7f4daaffd640 1 -- 172.21.15.31:0/457598716 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4d9c01a400 con 0x7f4db40ca310 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.415+0000 7f4daaffd640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d9406af00 0x7f4d9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.415+0000 7f4daaffd640 1 -- 172.21.15.31:0/457598716 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4d9c089870 con 0x7f4db40ca310 2024-01-19T04:56:05.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.415+0000 7f4db259c640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d9406af00 0x7f4d9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.416+0000 7f4db259c640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d9406af00 0x7f4d9406d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006fd0 tx=0x7f4da4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.419+0000 7f4daaffd640 1 -- 172.21.15.31:0/457598716 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4d9c054630 con 0x7f4db40ca310 2024-01-19T04:56:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.638+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4db40af4b0 con 0x7f4db40ca310 2024-01-19T04:56:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.638+0000 7f4daaffd640 1 -- 172.21.15.31:0/457598716 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f4d9c015070 con 0x7f4db40ca310 2024-01-19T04:56:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:05.638 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:56:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:56:05.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d9406af00 msgr2=0x7f4d9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d9406af00 0x7f4d9406d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006fd0 tx=0x7f4da4008040 comp rx=0 tx=0).stop 2024-01-19T04:56:05.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db40ca310 msgr2=0x7f4db40ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db40ca310 0x7f4db40ca960 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00b3e0 tx=0x7f4d9c00b8a0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 shutdown_connections 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4d9406af00 0x7f4d9406d3c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40cba90 0x7f4db414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db40ca310 0x7f4db40ca960 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.640+0000 7f4db8f68640 1 --2- 172.21.15.31:0/457598716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db406b6d0 0x7f4db4150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.641+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 >> 172.21.15.31:0/457598716 conn(0x7f4db40cc1b0 msgr2=0x7f4db40cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.641+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 shutdown_connections 2024-01-19T04:56:05.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.641+0000 7f4db8f68640 1 -- 172.21.15.31:0/457598716 wait complete. 2024-01-19T04:56:05.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.752+0000 7fc28e5ea640 1 Processor -- start 2024-01-19T04:56:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.752+0000 7fc28e5ea640 1 -- start start 2024-01-19T04:56:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.753+0000 7fc28e5ea640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 0x7fc28815c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.753+0000 7fc28e5ea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880d0040 con 0x7fc28806ecb0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.753+0000 7fc28e5ea640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28815c800 con 0x7fc28806e8d0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.753+0000 7fc28e5ea640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880cd500 con 0x7fc2880d08a0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.753+0000 7fc28cb87640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc28806ecb0 0x7fc2880cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58466/0 (socket says 172.21.15.31:58466) 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.753+0000 7fc28cb87640 1 -- 172.21.15.31:0/1191629530 learned_addr learned my addr 172.21.15.31:0/1191629530 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.753+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045892529 0 0) 0x7fc2880d0040 con 0x7fc28806ecb0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc274003520 con 0x7fc28806ecb0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc2877fe640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 0x7fc28815c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1465562545 0 0) 0x7fc274003520 con 0x7fc28806ecb0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 msgr2=0x7fc28815c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 0x7fc28815c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 >> v1:172.21.15.137:6789/0 conn(0x7fc28806e8d0 legacy=0x7fc2880cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880cb800 con 0x7fc28806ecb0 2024-01-19T04:56:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fc2800019f0 con 0x7fc28806ecb0 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc2877fe640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 0x7fc28815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 >> v1:172.21.15.31:6789/0 conn(0x7fc28806ecb0 legacy=0x7fc2880cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 0x7fc2740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274003f90 0x7fc2740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274004d00 0x7fc274005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880cd500 con 0x7fc274004d00 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400bb70 con 0x7fc274003f90 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400bd30 con 0x7fc2880d08a0 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc28cb87640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274004d00 0x7fc274005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc287fff640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 0x7fc2740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc2877fe640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274003f90 0x7fc2740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc28cb87640 1 -- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 msgr2=0x7fc2740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc28cb87640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2880d08a0 0x7fc2740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc28cb87640 1 -- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274003f90 msgr2=0x7fc2740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc28cb87640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274003f90 0x7fc2740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.754+0000 7fc28cb87640 1 -- 172.21.15.31:0/1191629530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274003520 con 0x7fc274004d00 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.755+0000 7fc28cb87640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274004d00 0x7fc274005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc28000a830 tx=0x7fc2740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a7ddeccb18cf7c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.755+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2800072d0 con 0x7fc274004d00 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.755+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28002f070 con 0x7fc274004d00 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.755+0000 7fc286ffd640 1 -- 172.21.15.31:0/1191629530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2800076f0 con 0x7fc274004d00 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.756+0000 7fc28e5ea640 1 -- 172.21.15.31:0/1191629530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274004d00 msgr2=0x7fc274005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.756+0000 7fc28e5ea640 1 --2- 172.21.15.31:0/1191629530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274004d00 0x7fc274005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc28000a830 tx=0x7fc2740057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.756+0000 7fc28cb87640 1 -- 172.21.15.31:0/1191629530 reap_dead start 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.756+0000 7fc28e5ea640 1 -- 172.21.15.31:0/1191629530 shutdown_connections 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.756+0000 7fc28e5ea640 1 -- 172.21.15.31:0/1191629530 >> 172.21.15.31:0/1191629530 conn(0x7fc2880b2c60 msgr2=0x7fc2880b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.756+0000 7fc28e5ea640 1 -- 172.21.15.31:0/1191629530 shutdown_connections 2024-01-19T04:56:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.756+0000 7fc28e5ea640 1 -- 172.21.15.31:0/1191629530 wait complete. 2024-01-19T04:56:05.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.757+0000 7fc28e5ea640 1 Processor -- start 2024-01-19T04:56:05.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.757+0000 7fc28e5ea640 1 -- start start 2024-01-19T04:56:05.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28e5ea640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc28814e4f0 0x7fc28806c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28e5ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc28806cda0 0x7fc2880b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28e5ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2880b5c60 0x7fc28806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28e5ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c5070 con 0x7fc2880b5c60 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28e5ea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c5350 con 0x7fc28806cda0 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28e5ea640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c5630 con 0x7fc28814e4f0 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2880b5c60 0x7fc28806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2880b5c60 0x7fc28806b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40002/0 (socket says 172.21.15.31:40002) 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 -- 172.21.15.31:0/2999417894 learned_addr learned my addr 172.21.15.31:0/2999417894 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc2877fe640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc28806cda0 0x7fc2880b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc287fff640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc28814e4f0 0x7fc28806c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 -- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc28814e4f0 msgr2=0x7fc28806c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc28814e4f0 0x7fc28806c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 -- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc28806cda0 msgr2=0x7fc2880b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc28806cda0 0x7fc2880b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.758+0000 7fc28cb87640 1 -- 172.21.15.31:0/2999417894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880c5910 con 0x7fc2880b5c60 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.759+0000 7fc28cb87640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2880b5c60 0x7fc28806b6c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc280001cb0 tx=0x7fc280001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.759+0000 7fc284ff9640 1 -- 172.21.15.31:0/2999417894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280044070 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.759+0000 7fc284ff9640 1 -- 172.21.15.31:0/2999417894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc280001250 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.759+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2880c5bf0 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.759+0000 7fc284ff9640 1 -- 172.21.15.31:0/2999417894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2800074e0 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.759+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880c4cf0 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.760+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288074a80 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.760+0000 7fc284ff9640 1 -- 172.21.15.31:0/2999417894 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc280031030 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.760+0000 7fc284ff9640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc26806af00 0x7fc26806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.761+0000 7fc284ff9640 1 -- 172.21.15.31:0/2999417894 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc2800ab230 con 0x7fc2880b5c60 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.761+0000 7fc287fff640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc26806af00 0x7fc26806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:05.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.761+0000 7fc287fff640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc26806af00 0x7fc26806d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2700017e0 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:05.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.765+0000 7fc284ff9640 1 -- 172.21.15.31:0/2999417894 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc2800761c0 con 0x7fc2880b5c60 2024-01-19T04:56:05.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.946+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc288074a80 con 0x7fc2880b5c60 2024-01-19T04:56:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.947+0000 7fc284ff9640 1 -- 172.21.15.31:0/2999417894 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16720 (secure 0 0 0) 0x7fc2800797e0 con 0x7fc2880b5c60 2024-01-19T04:56:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:05.947 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:01.578429+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc26806af00 msgr2=0x7fc26806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc26806af00 0x7fc26806d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2700017e0 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2880b5c60 msgr2=0x7fc28806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2880b5c60 0x7fc28806b6c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc280001cb0 tx=0x7fc280001d80 comp rx=0 tx=0).stop 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 shutdown_connections 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc26806af00 0x7fc26806d3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2880b5c60 0x7fc28806b6c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc28806cda0 0x7fc2880b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 --2- 172.21.15.31:0/2999417894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc28814e4f0 0x7fc28806c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:05.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 >> 172.21.15.31:0/2999417894 conn(0x7fc2880b2c60 msgr2=0x7fc2880b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:05.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 shutdown_connections 2024-01-19T04:56:05.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:05.949+0000 7fc28e5ea640 1 -- 172.21.15.31:0/2999417894 wait complete. 2024-01-19T04:56:05.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:55:16.957427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:55:17.841010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:05.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:06.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.065+0000 7f566012d640 1 Processor -- start 2024-01-19T04:56:06.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.065+0000 7f566012d640 1 -- start start 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f566012d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d18e0 0x7f565815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f566012d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56580d0ef0 0x7f56580cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f566012d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565815dee0 con 0x7f56580d3e50 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f566012d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565815e1c0 con 0x7f56580d0ef0 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f566012d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565815e4a0 con 0x7f56580d18e0 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565e6ca640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56580d0ef0 0x7f56580cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565e6ca640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56580d0ef0 0x7f56580cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40008/0 (socket says 172.21.15.31:40008) 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565e6ca640 1 -- 172.21.15.31:0/2164552853 learned_addr learned my addr 172.21.15.31:0/2164552853 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565dec9640 1 --2- 172.21.15.31:0/2164552853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d18e0 0x7f565815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565dec9640 1 -- 172.21.15.31:0/2164552853 >> v2:172.21.15.31:3300/0 conn(0x7f56580d0ef0 msgr2=0x7f56580cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:06.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565dec9640 1 --2- 172.21.15.31:0/2164552853 >> v2:172.21.15.31:3300/0 conn(0x7f56580d0ef0 0x7f56580cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565dec9640 1 -- 172.21.15.31:0/2164552853 >> v1:172.21.15.137:6789/0 conn(0x7f56580d3e50 legacy=0x7f56580d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565cec7640 1 -- 172.21.15.31:0/2164552853 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469691938 0 0) 0x7f565815dee0 con 0x7f56580d3e50 2024-01-19T04:56:06.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565dec9640 1 -- 172.21.15.31:0/2164552853 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565815e780 con 0x7f56580d18e0 2024-01-19T04:56:06.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.066+0000 7f565e6ca640 1 --2- 172.21.15.31:0/2164552853 >> v2:172.21.15.31:3300/0 conn(0x7f56580d0ef0 0x7f56580cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:06.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.067+0000 7f565dec9640 1 --2- 172.21.15.31:0/2164552853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d18e0 0x7f565815d860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f56540097a0 tx=0x7f565402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624a3173e0c212f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.067+0000 7f565cec7640 1 -- 172.21.15.31:0/2164552853 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565402d040 con 0x7f56580d18e0 2024-01-19T04:56:06.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.067+0000 7f565cec7640 1 -- 172.21.15.31:0/2164552853 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654031810 con 0x7f56580d18e0 2024-01-19T04:56:06.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f565cec7640 1 -- 172.21.15.31:0/2164552853 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654031ba0 con 0x7f56580d18e0 2024-01-19T04:56:06.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 -- 172.21.15.31:0/2164552853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d18e0 msgr2=0x7f565815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 --2- 172.21.15.31:0/2164552853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d18e0 0x7f565815d860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f56540097a0 tx=0x7f565402b8b0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 -- 172.21.15.31:0/2164552853 shutdown_connections 2024-01-19T04:56:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 --2- 172.21.15.31:0/2164552853 >> v2:172.21.15.31:3300/0 conn(0x7f56580d0ef0 0x7f56580cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 --2- 172.21.15.31:0/2164552853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d18e0 0x7f565815d860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 -- 172.21.15.31:0/2164552853 >> 172.21.15.31:0/2164552853 conn(0x7f565809c600 msgr2=0x7f56580d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 -- 172.21.15.31:0/2164552853 shutdown_connections 2024-01-19T04:56:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.068+0000 7f566012d640 1 -- 172.21.15.31:0/2164552853 wait complete. 2024-01-19T04:56:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.069+0000 7f566012d640 1 Processor -- start 2024-01-19T04:56:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.069+0000 7f566012d640 1 -- start start 2024-01-19T04:56:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f566012d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d0ef0 0x7f5658149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f566012d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56580d18e0 0x7f565815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f566012d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56580d3e50 0x7f56580cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f566012d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580ca610 con 0x7f56580d3e50 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f566012d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581f7540 con 0x7f56580d18e0 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f566012d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565814a140 con 0x7f56580d0ef0 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56580d3e50 0x7f56580cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56580d3e50 0x7f56580cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40018/0 (socket says 172.21.15.31:40018) 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 -- 172.21.15.31:0/3409312507 learned_addr learned my addr 172.21.15.31:0/3409312507 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565dec9640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d0ef0 0x7f5658149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565d6c8640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56580d18e0 0x7f565815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 -- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d0ef0 msgr2=0x7f5658149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d0ef0 0x7f5658149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 -- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56580d18e0 msgr2=0x7f565815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56580d18e0 0x7f565815a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.070+0000 7f565e6ca640 1 -- 172.21.15.31:0/3409312507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650009b30 con 0x7f56580d3e50 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.071+0000 7f565e6ca640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56580d3e50 0x7f56580cca30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5650009f10 tx=0x7f565000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.071+0000 7f564e7fc640 1 -- 172.21.15.31:0/3409312507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650018070 con 0x7f56580d3e50 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.071+0000 7f564e7fc640 1 -- 172.21.15.31:0/3409312507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565000f040 con 0x7f56580d3e50 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.071+0000 7f564e7fc640 1 -- 172.21.15.31:0/3409312507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565000ce60 con 0x7f56580d3e50 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.071+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5654009400 con 0x7f56580d3e50 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.071+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56580cc5a0 con 0x7f56580d3e50 2024-01-19T04:56:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.072+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56580cc730 con 0x7f56580d3e50 2024-01-19T04:56:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.072+0000 7f564e7fc640 1 -- 172.21.15.31:0/3409312507 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5650014480 con 0x7f56580d3e50 2024-01-19T04:56:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.073+0000 7f564e7fc640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f563c06af00 0x7f563c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.073+0000 7f564e7fc640 1 -- 172.21.15.31:0/3409312507 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f56500897a0 con 0x7f56580d3e50 2024-01-19T04:56:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.073+0000 7f565dec9640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f563c06af00 0x7f563c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.074+0000 7f565dec9640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f563c06af00 0x7f563c06d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5654036000 tx=0x7f565403d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.077+0000 7f564e7fc640 1 -- 172.21.15.31:0/3409312507 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5650054730 con 0x7f56580d3e50 2024-01-19T04:56:06.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.253+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5658074900 con 0x7f56580d3e50 2024-01-19T04:56:06.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.254+0000 7f564e7fc640 1 -- 172.21.15.31:0/3409312507 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5650057d50 con 0x7f56580d3e50 2024-01-19T04:56:06.253 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:06.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f563c06af00 msgr2=0x7f563c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f563c06af00 0x7f563c06d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5654036000 tx=0x7f565403d040 comp rx=0 tx=0).stop 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56580d3e50 msgr2=0x7f56580cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56580d3e50 0x7f56580cca30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5650009f10 tx=0x7f565000c490 comp rx=0 tx=0).stop 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 shutdown_connections 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f563c06af00 0x7f563c06d3c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56580d3e50 0x7f56580cca30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56580d18e0 0x7f565815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 --2- 172.21.15.31:0/3409312507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f56580d0ef0 0x7f5658149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 >> 172.21.15.31:0/3409312507 conn(0x7f565809c600 msgr2=0x7f565806bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 shutdown_connections 2024-01-19T04:56:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.256+0000 7f566012d640 1 -- 172.21.15.31:0/3409312507 wait complete. 2024-01-19T04:56:06.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.370+0000 7f701299e640 1 Processor -- start 2024-01-19T04:56:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.370+0000 7f701299e640 1 -- start start 2024-01-19T04:56:06.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.371+0000 7f701299e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.371+0000 7f701299e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0cf2b0 con 0x7f700c0d1090 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.371+0000 7f701299e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0cea10 con 0x7f700c0d1be0 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.371+0000 7f701299e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0cecf0 con 0x7f700c160bc0 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.371+0000 7f701119b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f700c0d1090 0x7f700c0d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58472/0 (socket says 172.21.15.31:58472) 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.371+0000 7f701119b640 1 -- 172.21.15.31:0/832261337 learned_addr learned my addr 172.21.15.31:0/832261337 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701199c640 1 --2- 172.21.15.31:0/832261337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182119046 0 0) 0x7f700c0cf2b0 con 0x7f700c0d1090 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff8003520 con 0x7f700c0d1090 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468985221 0 0) 0x7f700c0cea10 con 0x7f700c0d1be0 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f700c0cf2b0 con 0x7f700c0d1be0 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1825677505 0 0) 0x7f6ff8003520 con 0x7f700c0d1090 2024-01-19T04:56:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701199c640 1 -- 172.21.15.31:0/832261337 >> v1:172.21.15.137:6789/0 conn(0x7f700c0d1be0 legacy=0x7f700c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701199c640 1 -- 172.21.15.31:0/832261337 >> v1:172.21.15.31:6789/0 conn(0x7f700c0d1090 legacy=0x7f700c0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.372+0000 7f701199c640 1 -- 172.21.15.31:0/832261337 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0cefd0 con 0x7f700c160bc0 2024-01-19T04:56:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701199c640 1 --2- 172.21.15.31:0/832261337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c0d3c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f70040097d0 tx=0x7f700402b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c009a37971738172 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc001670 con 0x7f700c160bc0 2024-01-19T04:56:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700402e050 con 0x7f700c160bc0 2024-01-19T04:56:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701099a640 1 -- 172.21.15.31:0/832261337 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004030710 con 0x7f700c160bc0 2024-01-19T04:56:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701299e640 1 -- 172.21.15.31:0/832261337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 msgr2=0x7f700c0d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701299e640 1 --2- 172.21.15.31:0/832261337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c0d3c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f70040097d0 tx=0x7f700402b6e0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701299e640 1 -- 172.21.15.31:0/832261337 shutdown_connections 2024-01-19T04:56:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701299e640 1 --2- 172.21.15.31:0/832261337 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c0d3c80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.373+0000 7f701299e640 1 -- 172.21.15.31:0/832261337 >> 172.21.15.31:0/832261337 conn(0x7f700c0d3190 msgr2=0x7f700c0d3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.374+0000 7f701299e640 1 -- 172.21.15.31:0/832261337 shutdown_connections 2024-01-19T04:56:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.374+0000 7f701299e640 1 -- 172.21.15.31:0/832261337 wait complete. 2024-01-19T04:56:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.375+0000 7f701299e640 1 Processor -- start 2024-01-19T04:56:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.375+0000 7f701299e640 1 -- start start 2024-01-19T04:56:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.375+0000 7f701299e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1090 0x7f700c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.375+0000 7f701299e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1be0 0x7f700c0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.375+0000 7f701299e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701299e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0b0510 con 0x7f700c0d1090 2024-01-19T04:56:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701299e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0cc300 con 0x7f700c0d1be0 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701299e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0cc5e0 con 0x7f700c160bc0 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1090 0x7f700c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1090 0x7f700c0b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40034/0 (socket says 172.21.15.31:40034) 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 -- 172.21.15.31:0/3065614865 learned_addr learned my addr 172.21.15.31:0/3065614865 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701119b640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1be0 0x7f700c0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701219d640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 -- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 msgr2=0x7f700c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 -- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1be0 msgr2=0x7f700c0cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1be0 0x7f700c0cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 -- 172.21.15.31:0/3065614865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7004009400 con 0x7f700c0d1090 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701199c640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1090 0x7f700c0b09a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f700402b720 tx=0x7f70040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f70027fc640 1 -- 172.21.15.31:0/3065614865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004003ea0 con 0x7f700c0d1090 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f70027fc640 1 -- 172.21.15.31:0/3065614865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7004030730 con 0x7f700c0d1090 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c0afec0 con 0x7f700c0d1090 2024-01-19T04:56:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f70027fc640 1 -- 172.21.15.31:0/3065614865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004030ae0 con 0x7f700c0d1090 2024-01-19T04:56:06.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.376+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c0cbf10 con 0x7f700c0d1090 2024-01-19T04:56:06.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.377+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c074900 con 0x7f700c0d1090 2024-01-19T04:56:06.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.378+0000 7f70027fc640 1 -- 172.21.15.31:0/3065614865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7004030c40 con 0x7f700c0d1090 2024-01-19T04:56:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.378+0000 7f70027fc640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff006aed0 0x7f6ff006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.378+0000 7f70027fc640 1 -- 172.21.15.31:0/3065614865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f70040ab2e0 con 0x7f700c0d1090 2024-01-19T04:56:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.378+0000 7f701119b640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff006aed0 0x7f6ff006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.379+0000 7f701119b640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff006aed0 0x7f6ff006d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f700c0d1890 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.383+0000 7f70027fc640 1 -- 172.21.15.31:0/3065614865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f70040760b0 con 0x7f700c0d1090 2024-01-19T04:56:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.560+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f700c0d0400 con 0x7f700c0d1090 2024-01-19T04:56:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.561+0000 7f70027fc640 1 -- 172.21.15.31:0/3065614865 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70040796d0 con 0x7f700c0d1090 2024-01-19T04:56:06.560 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:06.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff006aed0 msgr2=0x7f6ff006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff006aed0 0x7f6ff006d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f700c0d1890 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2024-01-19T04:56:06.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1090 msgr2=0x7f700c0b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1090 0x7f700c0b09a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f700402b720 tx=0x7f70040046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 shutdown_connections 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff006aed0 0x7f6ff006d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c160bc0 0x7f700c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1be0 0x7f700c0cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 --2- 172.21.15.31:0/3065614865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1090 0x7f700c0b09a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.562+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 >> 172.21.15.31:0/3065614865 conn(0x7f700c0d3190 msgr2=0x7f700c0b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.563+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 shutdown_connections 2024-01-19T04:56:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.563+0000 7f701299e640 1 -- 172.21.15.31:0/3065614865 wait complete. 2024-01-19T04:56:06.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.677+0000 7ff1965b6640 1 Processor -- start 2024-01-19T04:56:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.677+0000 7ff1965b6640 1 -- start start 2024-01-19T04:56:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff1965b6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 0x7ff19015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff1965b6640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19015dee0 con 0x7ff1900d0ef0 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff1965b6640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19015e1c0 con 0x7ff1900d18e0 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff1965b6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19015e4a0 con 0x7ff1900d3e50 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff194b53640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff1900d0ef0 0x7ff1900cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58480/0 (socket says 172.21.15.31:58480) 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff194b53640 1 -- 172.21.15.31:0/2438072341 learned_addr learned my addr 172.21.15.31:0/2438072341 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff18f7fe640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 0x7ff19015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728646200 0 0) 0x7ff19015dee0 con 0x7ff1900d0ef0 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.678+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff178003520 con 0x7ff1900d0ef0 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4140787057 0 0) 0x7ff178003520 con 0x7ff1900d0ef0 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 msgr2=0x7ff19015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 0x7ff19015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 >> v1:172.21.15.137:6789/0 conn(0x7ff1900d18e0 legacy=0x7ff1900d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19015e780 con 0x7ff1900d0ef0 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7ff1880019f0 con 0x7ff1900d0ef0 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 >> v1:172.21.15.31:6789/0 conn(0x7ff1900d0ef0 legacy=0x7ff1900cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780039a0 0x7ff178003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 0x7ff1780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff178005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19015e4a0 con 0x7ff178004d00 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17800b980 con 0x7ff1780039a0 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17800bb40 con 0x7ff1900d3e50 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff194b53640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff178005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18ffff640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780039a0 0x7ff178003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff18f7fe640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 0x7ff1780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff194b53640 1 -- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 msgr2=0x7ff1780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:06.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff194b53640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff1900d3e50 0x7ff1780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff194b53640 1 -- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780039a0 msgr2=0x7ff178003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:06.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff194b53640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780039a0 0x7ff178003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.679+0000 7ff194b53640 1 -- 172.21.15.31:0/2438072341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff178003520 con 0x7ff178004d00 2024-01-19T04:56:06.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.680+0000 7ff194b53640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff178005180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff18800a830 tx=0x7ff1780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4a74a71b3b45e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.680+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18802cea0 con 0x7ff178004d00 2024-01-19T04:56:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.680+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18802f070 con 0x7ff178004d00 2024-01-19T04:56:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.680+0000 7ff18effd640 1 -- 172.21.15.31:0/2438072341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188034890 con 0x7ff178004d00 2024-01-19T04:56:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.680+0000 7ff1965b6640 1 -- 172.21.15.31:0/2438072341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 msgr2=0x7ff178005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.680+0000 7ff1965b6640 1 --2- 172.21.15.31:0/2438072341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff178005180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff18800a830 tx=0x7ff1780057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.680+0000 7ff194b53640 1 -- 172.21.15.31:0/2438072341 reap_dead start 2024-01-19T04:56:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.681+0000 7ff1965b6640 1 -- 172.21.15.31:0/2438072341 shutdown_connections 2024-01-19T04:56:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.681+0000 7ff1965b6640 1 -- 172.21.15.31:0/2438072341 >> 172.21.15.31:0/2438072341 conn(0x7ff19009c600 msgr2=0x7ff1900d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.681+0000 7ff1965b6640 1 -- 172.21.15.31:0/2438072341 shutdown_connections 2024-01-19T04:56:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.681+0000 7ff1965b6640 1 -- 172.21.15.31:0/2438072341 wait complete. 2024-01-19T04:56:06.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.682+0000 7ff1965b6640 1 Processor -- start 2024-01-19T04:56:06.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.682+0000 7ff1965b6640 1 -- start start 2024-01-19T04:56:06.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff19014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1900b5e60 0x7ff19015fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff190160360 0x7ff19014ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900b89e0 con 0x7ff178004d00 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900b8c80 con 0x7ff1900b5e60 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19015f900 con 0x7ff190160360 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff19014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff19014a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40056/0 (socket says 172.21.15.31:40056) 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 -- 172.21.15.31:0/2625238605 learned_addr learned my addr 172.21.15.31:0/2625238605 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff194b53640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff190160360 0x7ff19014ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18f7fe640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1900b5e60 0x7ff19015fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 -- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff190160360 msgr2=0x7ff19014ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff190160360 0x7ff19014ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 -- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1900b5e60 msgr2=0x7ff19015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1900b5e60 0x7ff19015fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 -- 172.21.15.31:0/2625238605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900b62f0 con 0x7ff178004d00 2024-01-19T04:56:06.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff194b53640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff190160360 0x7ff19014ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18ffff640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff19014a0b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff18400f7c0 tx=0x7ff18400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18cff9640 1 -- 172.21.15.31:0/2625238605 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400fed0 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18cff9640 1 -- 172.21.15.31:0/2625238605 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1840052e0 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff18cff9640 1 -- 172.21.15.31:0/2625238605 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400d970 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff190150ce0 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.683+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19014a5f0 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.684+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19015f720 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.685+0000 7ff18cff9640 1 -- 172.21.15.31:0/2625238605 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff18400dad0 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.685+0000 7ff18cff9640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff17c06aed0 0x7ff17c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.685+0000 7ff18cff9640 1 -- 172.21.15.31:0/2625238605 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff184089770 con 0x7ff178004d00 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.685+0000 7ff18f7fe640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff17c06aed0 0x7ff17c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.686+0000 7ff18f7fe640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff17c06aed0 0x7ff17c06d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1901f6be0 tx=0x7ff180008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.689+0000 7ff18cff9640 1 -- 172.21.15.31:0/2625238605 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff1840545c0 con 0x7ff178004d00 2024-01-19T04:56:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.867+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff190074900 con 0x7ff178004d00 2024-01-19T04:56:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.868+0000 7ff18cff9640 1 -- 172.21.15.31:0/2625238605 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff184057be0 con 0x7ff178004d00 2024-01-19T04:56:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.869+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff17c06aed0 msgr2=0x7ff17c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.869+0000 7ff1965b6640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff17c06aed0 0x7ff17c06d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1901f6be0 tx=0x7ff180008040 comp rx=0 tx=0).stop 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.869+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 msgr2=0x7ff19014a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.869+0000 7ff1965b6640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff19014a0b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff18400f7c0 tx=0x7ff18400fc80 comp rx=0 tx=0).stop 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 shutdown_connections 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff17c06aed0 0x7ff17c06d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff190160360 0x7ff19014ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1900b5e60 0x7ff19015fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 --2- 172.21.15.31:0/2625238605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff178004d00 0x7ff19014a0b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 >> 172.21.15.31:0/2625238605 conn(0x7ff19009c600 msgr2=0x7ff1900b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 shutdown_connections 2024-01-19T04:56:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.870+0000 7ff1965b6640 1 -- 172.21.15.31:0/2625238605 wait complete. 2024-01-19T04:56:06.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.981+0000 7f9aeab51640 1 Processor -- start 2024-01-19T04:56:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.981+0000 7f9aeab51640 1 -- start start 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9aeab51640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9aeab51640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9ae415f0c0 0x7f9ae415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9aeab51640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae406bf30 con 0x7f9ae415eb70 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9aeab51640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae415fa10 con 0x7f9ae415f0c0 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9aeab51640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40cb9e0 con 0x7f9ae415d8a0 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9ae90ee640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9ae415f0c0 0x7f9ae415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9ae88ed640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9ae90ee640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9ae415f0c0 0x7f9ae415f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:50426/0 (socket says 172.21.15.31:50426) 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9ae88ed640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae406b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:36584/0 (socket says 172.21.15.31:36584) 2024-01-19T04:56:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1733266777 learned_addr learned my addr 172.21.15.31:0/1733266777 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.982+0000 7f9ae3fff640 1 --1- 172.21.15.31:0/1733266777 >> v1:172.21.15.137:6789/0 conn(0x7f9ae415eb70 0x7f9ae40cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:59990/0 (socket says 172.21.15.31:59990) 2024-01-19T04:56:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.983+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1733266777 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 msgr2=0x7f9ae406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.983+0000 7f9ae90ee640 1 --2- 172.21.15.31:0/1733266777 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae406b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.983+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1733266777 >> v1:172.21.15.137:6789/0 conn(0x7f9ae415eb70 legacy=0x7f9ae40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.983+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1733266777 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40cbcc0 con 0x7f9ae415f0c0 2024-01-19T04:56:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.983+0000 7f9ae90ee640 1 --2- 172.21.15.31:0/1733266777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415f0c0 0x7f9ae415f4d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009870 tx=0x7f9acc02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44aa259e6a7006b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9ae37fe640 1 -- 172.21.15.31:0/1733266777 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc031640 con 0x7f9ae415f0c0 2024-01-19T04:56:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9ae37fe640 1 -- 172.21.15.31:0/1733266777 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc0317a0 con 0x7f9ae415f0c0 2024-01-19T04:56:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9ae37fe640 1 -- 172.21.15.31:0/1733266777 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc031b30 con 0x7f9ae415f0c0 2024-01-19T04:56:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 -- 172.21.15.31:0/1733266777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415f0c0 msgr2=0x7f9ae415f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1733266777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415f0c0 0x7f9ae415f4d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009870 tx=0x7f9acc02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 -- 172.21.15.31:0/1733266777 shutdown_connections 2024-01-19T04:56:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1733266777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415f0c0 0x7f9ae415f4d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1733266777 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae406b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 -- 172.21.15.31:0/1733266777 >> 172.21.15.31:0/1733266777 conn(0x7f9ae406abe0 msgr2=0x7f9ae4066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 -- 172.21.15.31:0/1733266777 shutdown_connections 2024-01-19T04:56:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.984+0000 7f9aeab51640 1 -- 172.21.15.31:0/1733266777 wait complete. 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.985+0000 7f9aeab51640 1 Processor -- start 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.985+0000 7f9aeab51640 1 -- start start 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9aeab51640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae414b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9aeab51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415eb70 0x7f9ae40b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9aeab51640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae415f0c0 0x7f9ae40b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9aeab51640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b6c00 con 0x7f9ae415f0c0 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9aeab51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c6eb0 con 0x7f9ae415eb70 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9aeab51640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c7190 con 0x7f9ae415d8a0 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9ae90ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae415f0c0 0x7f9ae40b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9ae90ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae415f0c0 0x7f9ae40b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40070/0 (socket says 172.21.15.31:40070) 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.986+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1946816976 learned_addr learned my addr 172.21.15.31:0/1946816976 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae88ed640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae414b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae3fff640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415eb70 0x7f9ae40b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 msgr2=0x7f9ae414b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae90ee640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae414b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415eb70 msgr2=0x7f9ae40b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae90ee640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415eb70 0x7f9ae40b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae90ee640 1 -- 172.21.15.31:0/1946816976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0094d0 con 0x7f9ae415f0c0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae90ee640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae415f0c0 0x7f9ae40b19f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009870 tx=0x7f9acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae17fa640 1 -- 172.21.15.31:0/1946816976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc03a930 con 0x7f9ae415f0c0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae17fa640 1 -- 172.21.15.31:0/1946816976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc03aab0 con 0x7f9ae415f0c0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9ae17fa640 1 -- 172.21.15.31:0/1946816976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc03ae60 con 0x7f9ae415f0c0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae40c7470 con 0x7f9ae415f0c0 2024-01-19T04:56:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.987+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae40c7a30 con 0x7f9ae415f0c0 2024-01-19T04:56:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.988+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae40c7c20 con 0x7f9ae415f0c0 2024-01-19T04:56:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.989+0000 7f9ae17fa640 1 -- 172.21.15.31:0/1946816976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9acc02f050 con 0x7f9ae415f0c0 2024-01-19T04:56:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.989+0000 7f9ae17fa640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ac406aed0 0x7f9ac406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.989+0000 7f9ae17fa640 1 -- 172.21.15.31:0/1946816976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9acc0ab520 con 0x7f9ae415f0c0 2024-01-19T04:56:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.989+0000 7f9ae88ed640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ac406aed0 0x7f9ac406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.990+0000 7f9ae88ed640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ac406aed0 0x7f9ac406d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009b40 tx=0x7f9adc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:06.993+0000 7f9ae17fa640 1 -- 172.21.15.31:0/1946816976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9acc0764b0 con 0x7f9ae415f0c0 2024-01-19T04:56:07.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.214+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ae40696e0 con 0x7f9ae415f0c0 2024-01-19T04:56:07.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.215+0000 7f9ae17fa640 1 -- 172.21.15.31:0/1946816976 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f9acc079ad0 con 0x7f9ae415f0c0 2024-01-19T04:56:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:07.215 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:56:07.215 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ac406aed0 msgr2=0x7f9ac406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ac406aed0 0x7f9ac406d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009b40 tx=0x7f9adc009190 comp rx=0 tx=0).stop 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae415f0c0 msgr2=0x7f9ae40b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae415f0c0 0x7f9ae40b19f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009870 tx=0x7f9acc0046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 shutdown_connections 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9ac406aed0 0x7f9ac406d390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae415f0c0 0x7f9ae40b19f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae415eb70 0x7f9ae40b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 --2- 172.21.15.31:0/1946816976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9ae415d8a0 0x7f9ae414b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 >> 172.21.15.31:0/1946816976 conn(0x7f9ae406abe0 msgr2=0x7f9ae40b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 shutdown_connections 2024-01-19T04:56:07.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.217+0000 7f9aeab51640 1 -- 172.21.15.31:0/1946816976 wait complete. 2024-01-19T04:56:07.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.328+0000 7fa411fca640 1 Processor -- start 2024-01-19T04:56:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.328+0000 7fa411fca640 1 -- start start 2024-01-19T04:56:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.329+0000 7fa411fca640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa40c0d2e50 0x7fa40c0b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.329+0000 7fa411fca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 0x7fa40c0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.329+0000 7fa411fca640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c160df0 con 0x7fa40c160a10 2024-01-19T04:56:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.329+0000 7fa411fca640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0b7300 con 0x7fa40c0d2e50 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.329+0000 7fa411fca640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0b5680 con 0x7fa40c0b4d00 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.329+0000 7fa40b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa40c160a10 0x7fa40c0d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58496/0 (socket says 172.21.15.31:58496) 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.329+0000 7fa40b7fe640 1 -- 172.21.15.31:0/3336264504 learned_addr learned my addr 172.21.15.31:0/3336264504 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40affd640 1 --2- 172.21.15.31:0/3336264504 >> v2:172.21.15.137:3300/0 conn(0x7fa40c0d2e50 0x7fa40c0b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40bfff640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 0x7fa40c0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865638274 0 0) 0x7fa40c160df0 con 0x7fa40c160a10 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f4003520 con 0x7fa40c160a10 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1877215946 0 0) 0x7fa3f4003520 con 0x7fa40c160a10 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 msgr2=0x7fa40c0b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 0x7fa40c0b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 >> v2:172.21.15.137:3300/0 conn(0x7fa40c0d2e50 msgr2=0x7fa40c0b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 --2- 172.21.15.31:0/3336264504 >> v2:172.21.15.137:3300/0 conn(0x7fa40c0d2e50 0x7fa40c0b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0d1db0 con 0x7fa40c160a10 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fa3fc001670 con 0x7fa40c160a10 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 >> v1:172.21.15.31:6789/0 conn(0x7fa40c160a10 legacy=0x7fa40c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 0x7fa3f4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa3f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f4004d00 0x7fa3f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0b7300 con 0x7fa3f4003ef0 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0b5680 con 0x7fa3f4004d00 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f400b9b0 con 0x7fa40c0b4d00 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40affd640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa3f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40b7fe640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 0x7fa3f4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40bfff640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f4004d00 0x7fa3f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40affd640 1 -- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 msgr2=0x7fa3f4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40affd640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0b4d00 0x7fa3f4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40affd640 1 -- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f4004d00 msgr2=0x7fa3f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40affd640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f4004d00 0x7fa3f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40affd640 1 -- 172.21.15.31:0/3336264504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f4003520 con 0x7fa3f4003ef0 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.330+0000 7fa40bfff640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3f4004d00 0x7fa3f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.331+0000 7fa40affd640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa3f40047c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0cfba0 tx=0x7fa3f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c8fe744451f5e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.331+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40002c680 con 0x7fa3f4003ef0 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.331+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40002c7e0 con 0x7fa3f4003ef0 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.331+0000 7fa40a7fc640 1 -- 172.21.15.31:0/3336264504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40002cb90 con 0x7fa3f4003ef0 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.332+0000 7fa411fca640 1 -- 172.21.15.31:0/3336264504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 msgr2=0x7fa3f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.332+0000 7fa411fca640 1 --2- 172.21.15.31:0/3336264504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa3f40047c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0cfba0 tx=0x7fa3f40057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.332+0000 7fa40bfff640 1 -- 172.21.15.31:0/3336264504 reap_dead start 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.332+0000 7fa411fca640 1 -- 172.21.15.31:0/3336264504 shutdown_connections 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.332+0000 7fa411fca640 1 -- 172.21.15.31:0/3336264504 >> 172.21.15.31:0/3336264504 conn(0x7fa40c072310 msgr2=0x7fa40c0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.332+0000 7fa411fca640 1 -- 172.21.15.31:0/3336264504 shutdown_connections 2024-01-19T04:56:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.332+0000 7fa411fca640 1 -- 172.21.15.31:0/3336264504 wait complete. 2024-01-19T04:56:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.333+0000 7fa411fca640 1 Processor -- start 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.333+0000 7fa411fca640 1 -- start start 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa411fca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa40c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa411fca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c158820 0x7fa40c14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa411fca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0ccb60 0x7fa40c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa411fca640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0cb870 con 0x7fa3f4003ef0 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa411fca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0cbac0 con 0x7fa40c158820 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa411fca640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0cbda0 con 0x7fa40c0ccb60 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa40c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa40c15a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40086/0 (socket says 172.21.15.31:40086) 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 -- 172.21.15.31:0/2064938845 learned_addr learned my addr 172.21.15.31:0/2064938845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 -- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0ccb60 msgr2=0x7fa40c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40bfff640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0ccb60 0x7fa40c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40affd640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c158820 0x7fa40c14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0ccb60 0x7fa40c0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 -- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c158820 msgr2=0x7fa40c14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c158820 0x7fa40c14d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.334+0000 7fa40b7fe640 1 -- 172.21.15.31:0/2064938845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0cc080 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.335+0000 7fa40b7fe640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa40c15a010 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc000fb0 tx=0x7fa3fc0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.335+0000 7fa3f3fff640 1 -- 172.21.15.31:0/2064938845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc007e60 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.335+0000 7fa3f3fff640 1 -- 172.21.15.31:0/2064938845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc00f9c0 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.335+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c0cc360 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.335+0000 7fa3f3fff640 1 -- 172.21.15.31:0/2064938845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00fd70 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.335+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c15a550 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.336+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c0696e0 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.336+0000 7fa3f3fff640 1 -- 172.21.15.31:0/2064938845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa3fc01d460 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.336+0000 7fa3f3fff640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3f806af00 0x7fa3f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.337+0000 7fa3f3fff640 1 -- 172.21.15.31:0/2064938845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa3fc089830 con 0x7fa3f4003ef0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.337+0000 7fa40affd640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3f806af00 0x7fa3f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.337+0000 7fa40affd640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3f806af00 0x7fa3f806d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa400036000 tx=0x7fa400030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.341+0000 7fa3f3fff640 1 -- 172.21.15.31:0/2064938845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa3fc054680 con 0x7fa3f4003ef0 2024-01-19T04:56:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.520+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa40c15f4e0 con 0x7fa3f4003ef0 2024-01-19T04:56:07.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.521+0000 7fa3f3fff640 1 -- 172.21.15.31:0/2064938845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16720 (secure 0 0 0) 0x7fa3fc057ca0 con 0x7fa3f4003ef0 2024-01-19T04:56:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:07.521 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:01.578429+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3f806af00 msgr2=0x7fa3f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3f806af00 0x7fa3f806d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa400036000 tx=0x7fa400030000 comp rx=0 tx=0).stop 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 msgr2=0x7fa40c15a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa40c15a010 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc000fb0 tx=0x7fa3fc0077a0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 shutdown_connections 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa3f806af00 0x7fa3f806d3c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa40c0ccb60 0x7fa40c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c158820 0x7fa40c14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 --2- 172.21.15.31:0/2064938845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3f4003ef0 0x7fa40c15a010 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 >> 172.21.15.31:0/2064938845 conn(0x7fa40c072310 msgr2=0x7fa40c0d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 shutdown_connections 2024-01-19T04:56:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.523+0000 7fa411fca640 1 -- 172.21.15.31:0/2064938845 wait complete. 2024-01-19T04:56:07.534 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-19T04:56:07.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:07.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.636+0000 7f54b7e1e640 1 Processor -- start 2024-01-19T04:56:07.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.636+0000 7f54b7e1e640 1 -- start start 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b7e1e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54b00d1600 0x7f54b00d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b7e1e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b00d06b0 con 0x7f54b00cfd10 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b7e1e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b00d1b90 con 0x7f54b0160bb0 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b7e1e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b00d1e70 con 0x7f54b00d1600 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b63bb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54b00d1600 0x7f54b00d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b53b9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f54b00cfd10 0x7f54b00d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58498/0 (socket says 172.21.15.31:58498) 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b53b9640 1 -- 172.21.15.31:0/1120145809 learned_addr learned my addr 172.21.15.31:0/1120145809 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b63bb640 1 -- 172.21.15.31:0/1120145809 >> v1:172.21.15.137:6789/0 conn(0x7f54b0160bb0 legacy=0x7f54b00b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b63bb640 1 -- 172.21.15.31:0/1120145809 >> v1:172.21.15.31:6789/0 conn(0x7f54b00cfd10 legacy=0x7f54b00d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b63bb640 1 -- 172.21.15.31:0/1120145809 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00d2150 con 0x7f54b00d1600 2024-01-19T04:56:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.637+0000 7f54b4bb8640 1 -- 172.21.15.31:0/1120145809 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194889730 0 0) 0x7f54b00d06b0 con 0x7f54b00cfd10 2024-01-19T04:56:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.638+0000 7f54b63bb640 1 --2- 172.21.15.31:0/1120145809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b00d1600 0x7f54b00d0170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009d90 tx=0x7f54a802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d91d8f7ee3168bc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.638+0000 7f54b4bb8640 1 -- 172.21.15.31:0/1120145809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a802cd00 con 0x7f54b00d1600 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.638+0000 7f54b4bb8640 1 -- 172.21.15.31:0/1120145809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a802ce60 con 0x7f54b00d1600 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.638+0000 7f54b4bb8640 1 -- 172.21.15.31:0/1120145809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8034660 con 0x7f54b00d1600 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.638+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1120145809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b00d1600 msgr2=0x7f54b00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.638+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1120145809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b00d1600 0x7f54b00d0170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009d90 tx=0x7f54a802bf50 comp rx=0 tx=0).stop 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.639+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1120145809 shutdown_connections 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.639+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1120145809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b00d1600 0x7f54b00d0170 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.639+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1120145809 >> 172.21.15.31:0/1120145809 conn(0x7f54b009c600 msgr2=0x7f54b00d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.639+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1120145809 shutdown_connections 2024-01-19T04:56:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.639+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1120145809 wait complete. 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.640+0000 7f54b7e1e640 1 Processor -- start 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.640+0000 7f54b7e1e640 1 -- start start 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.640+0000 7f54b7e1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b00cfd10 0x7f54b00c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.640+0000 7f54b7e1e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54b00d1600 0x7f54b014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b7e1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b0160bb0 0x7f54b01f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b7e1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00c88f0 con 0x7f54b0160bb0 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b7e1e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b014dca0 con 0x7f54b00cfd10 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b7e1e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00b65e0 con 0x7f54b00d1600 2024-01-19T04:56:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b0160bb0 0x7f54b01f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b0160bb0 0x7f54b01f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40110/0 (socket says 172.21.15.31:40110) 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 -- 172.21.15.31:0/1247218383 learned_addr learned my addr 172.21.15.31:0/1247218383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b53b9640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54b00d1600 0x7f54b014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b5bba640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b00cfd10 0x7f54b00c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 -- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54b00d1600 msgr2=0x7f54b014d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54b00d1600 0x7f54b014d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 -- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b00cfd10 msgr2=0x7f54b00c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b00cfd10 0x7f54b00c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 -- 172.21.15.31:0/1247218383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8009a70 con 0x7f54b0160bb0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b63bb640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b0160bb0 0x7f54b01f3530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009d90 tx=0x7f54a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54a67fc640 1 -- 172.21.15.31:0/1247218383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a803d600 con 0x7f54b0160bb0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54a67fc640 1 -- 172.21.15.31:0/1247218383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a803d780 con 0x7f54b0160bb0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54a67fc640 1 -- 172.21.15.31:0/1247218383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a803db30 con 0x7f54b0160bb0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b00b68c0 con 0x7f54b0160bb0 2024-01-19T04:56:07.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.641+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b00b6e80 con 0x7f54b0160bb0 2024-01-19T04:56:07.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.642+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0067b50 con 0x7f54b0160bb0 2024-01-19T04:56:07.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.643+0000 7f54a67fc640 1 -- 172.21.15.31:0/1247218383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f54a803b040 con 0x7f54b0160bb0 2024-01-19T04:56:07.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.643+0000 7f54a67fc640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f549406aed0 0x7f549406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.643+0000 7f54a67fc640 1 -- 172.21.15.31:0/1247218383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f54a80abda0 con 0x7f54b0160bb0 2024-01-19T04:56:07.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.643+0000 7f54b5bba640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f549406aed0 0x7f549406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.644+0000 7f54b5bba640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f549406aed0 0x7f549406d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f54ac000f60 tx=0x7f54ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.647+0000 7f54a67fc640 1 -- 172.21.15.31:0/1247218383 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f54a8076c10 con 0x7f54b0160bb0 2024-01-19T04:56:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.861+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54b00696e0 con 0x7f54b0160bb0 2024-01-19T04:56:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.862+0000 7f54a67fc640 1 -- 172.21.15.31:0/1247218383 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f54a804e090 con 0x7f54b0160bb0 2024-01-19T04:56:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:56:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:56:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f549406aed0 msgr2=0x7f549406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f549406aed0 0x7f549406d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f54ac000f60 tx=0x7f54ac009040 comp rx=0 tx=0).stop 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b0160bb0 msgr2=0x7f54b01f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b0160bb0 0x7f54b01f3530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009d90 tx=0x7f54a8004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 shutdown_connections 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f549406aed0 0x7f549406d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54b0160bb0 0x7f54b01f3530 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54b00d1600 0x7f54b014d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 --2- 172.21.15.31:0/1247218383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b00cfd10 0x7f54b00c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 >> 172.21.15.31:0/1247218383 conn(0x7f54b009c600 msgr2=0x7f54b015d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 shutdown_connections 2024-01-19T04:56:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.864+0000 7f54b7e1e640 1 -- 172.21.15.31:0/1247218383 wait complete. 2024-01-19T04:56:07.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.979+0000 7fc371c9b640 1 Processor -- start 2024-01-19T04:56:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.979+0000 7fc371c9b640 1 -- start start 2024-01-19T04:56:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.979+0000 7fc371c9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc36c0b3030 0x7fc36c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.979+0000 7fc371c9b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc36c071df0 0x7fc36c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.979+0000 7fc371c9b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c0b3510 con 0x7fc36c15d300 2024-01-19T04:56:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.979+0000 7fc371c9b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c0b37f0 con 0x7fc36c0b3030 2024-01-19T04:56:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.979+0000 7fc371c9b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c0b3ad0 con 0x7fc36c071df0 2024-01-19T04:56:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc36c0b3030 0x7fc36c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc36c15d300 0x7fc36c06ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58502/0 (socket says 172.21.15.31:58502) 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36b7fe640 1 -- 172.21.15.31:0/1412891889 learned_addr learned my addr 172.21.15.31:0/1412891889 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36bfff640 1 --2- 172.21.15.31:0/1412891889 >> v2:172.21.15.137:3300/0 conn(0x7fc36c071df0 0x7fc36c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36affd640 1 -- 172.21.15.31:0/1412891889 >> v2:172.21.15.137:3300/0 conn(0x7fc36c071df0 msgr2=0x7fc36c1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36affd640 1 --2- 172.21.15.31:0/1412891889 >> v2:172.21.15.137:3300/0 conn(0x7fc36c071df0 0x7fc36c1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36affd640 1 -- 172.21.15.31:0/1412891889 >> v1:172.21.15.31:6789/0 conn(0x7fc36c15d300 legacy=0x7fc36c06ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36a7fc640 1 -- 172.21.15.31:0/1412891889 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205689492 0 0) 0x7fc36c0b3510 con 0x7fc36c15d300 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36affd640 1 -- 172.21.15.31:0/1412891889 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c06a650 con 0x7fc36c0b3030 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.980+0000 7fc36bfff640 1 --2- 172.21.15.31:0/1412891889 >> v2:172.21.15.137:3300/0 conn(0x7fc36c071df0 0x7fc36c1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc36affd640 1 --2- 172.21.15.31:0/1412891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b3030 0x7fc36c160f60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0b42d0 tx=0x7fc36002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6642e347cd257e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc36a7fc640 1 -- 172.21.15.31:0/1412891889 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36002e070 con 0x7fc36c0b3030 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc36a7fc640 1 -- 172.21.15.31:0/1412891889 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3600315a0 con 0x7fc36c0b3030 2024-01-19T04:56:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc36a7fc640 1 -- 172.21.15.31:0/1412891889 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360031930 con 0x7fc36c0b3030 2024-01-19T04:56:07.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 -- 172.21.15.31:0/1412891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b3030 msgr2=0x7fc36c160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 --2- 172.21.15.31:0/1412891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b3030 0x7fc36c160f60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0b42d0 tx=0x7fc36002b840 comp rx=0 tx=0).stop 2024-01-19T04:56:07.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 -- 172.21.15.31:0/1412891889 shutdown_connections 2024-01-19T04:56:07.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 --2- 172.21.15.31:0/1412891889 >> v2:172.21.15.137:3300/0 conn(0x7fc36c071df0 0x7fc36c1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 --2- 172.21.15.31:0/1412891889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b3030 0x7fc36c160f60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 -- 172.21.15.31:0/1412891889 >> 172.21.15.31:0/1412891889 conn(0x7fc36c06d750 msgr2=0x7fc36c06db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 -- 172.21.15.31:0/1412891889 shutdown_connections 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.981+0000 7fc371c9b640 1 -- 172.21.15.31:0/1412891889 wait complete. 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.983+0000 7fc371c9b640 1 Processor -- start 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.983+0000 7fc371c9b640 1 -- start start 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.983+0000 7fc371c9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc36c071df0 0x7fc36c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc371c9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc36c0b3030 0x7fc36c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc371c9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c15d300 0x7fc36c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc371c9b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0cf960 con 0x7fc36c15d300 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc371c9b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c150ec0 con 0x7fc36c0b3030 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc371c9b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1511a0 con 0x7fc36c071df0 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c15d300 0x7fc36c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c15d300 0x7fc36c15bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40114/0 (socket says 172.21.15.31:40114) 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 -- 172.21.15.31:0/508024877 learned_addr learned my addr 172.21.15.31:0/508024877 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:07.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36affd640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc36c0b3030 0x7fc36c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36b7fe640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc36c071df0 0x7fc36c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 -- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc36c071df0 msgr2=0x7fc36c150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc36c071df0 0x7fc36c150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 -- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc36c0b3030 msgr2=0x7fc36c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc36c0b3030 0x7fc36c0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 -- 172.21.15.31:0/508024877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360009400 con 0x7fc36c15d300 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36b7fe640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc36c071df0 0x7fc36c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc36bfff640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c15d300 0x7fc36c15bce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc36400cd80 tx=0x7fc36400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc353fff640 1 -- 172.21.15.31:0/508024877 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364003b90 con 0x7fc36c15d300 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc353fff640 1 -- 172.21.15.31:0/508024877 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364003d10 con 0x7fc36c15d300 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c151480 con 0x7fc36c15d300 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc353fff640 1 -- 172.21.15.31:0/508024877 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364004260 con 0x7fc36c15d300 2024-01-19T04:56:07.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.984+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c151af0 con 0x7fc36c15d300 2024-01-19T04:56:07.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.986+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c151d10 con 0x7fc36c15d300 2024-01-19T04:56:07.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.986+0000 7fc353fff640 1 -- 172.21.15.31:0/508024877 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc36400b040 con 0x7fc36c15d300 2024-01-19T04:56:07.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.986+0000 7fc353fff640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc35806aed0 0x7fc35806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:07.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.986+0000 7fc353fff640 1 -- 172.21.15.31:0/508024877 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc364089c50 con 0x7fc36c15d300 2024-01-19T04:56:07.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.987+0000 7fc36b7fe640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc35806aed0 0x7fc35806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:07.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.987+0000 7fc36b7fe640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc35806aed0 0x7fc35806d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000f60 tx=0x7fc35c007c10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:07.991+0000 7fc353fff640 1 -- 172.21.15.31:0/508024877 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc36408f020 con 0x7fc36c15d300 2024-01-19T04:56:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.206+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc36c0696e0 con 0x7fc36c15d300 2024-01-19T04:56:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.206+0000 7fc353fff640 1 -- 172.21.15.31:0/508024877 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fc36401d020 con 0x7fc36c15d300 2024-01-19T04:56:08.206 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:08.206 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3432724117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3432724117},{"type":"v1","addr":"172.21.15.137:6835","nonce":3432724117}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2844636627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2844636627},{"type":"v1","addr":"172.21.15.137:6837","nonce":2844636627}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3462568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3462568595},{"type":"v1","addr":"172.21.15.31:6835","nonce":3462568595}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:55:18.877117+0000","modified":"2024-01-19T04:55:20.870670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6833/2576238027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2576238027},{"type":"v1","addr":"172.21.15.31:6833","nonce":2576238027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-19T04:56:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc35806aed0 msgr2=0x7fc35806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc35806aed0 0x7fc35806d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000f60 tx=0x7fc35c007c10 comp rx=0 tx=0).stop 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c15d300 msgr2=0x7fc36c15bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c15d300 0x7fc36c15bce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc36400cd80 tx=0x7fc36400a650 comp rx=0 tx=0).stop 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 shutdown_connections 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc35806aed0 0x7fc35806d390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c15d300 0x7fc36c15bce0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc36c0b3030 0x7fc36c0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 --2- 172.21.15.31:0/508024877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc36c071df0 0x7fc36c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 >> 172.21.15.31:0/508024877 conn(0x7fc36c06d750 msgr2=0x7fc36c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 shutdown_connections 2024-01-19T04:56:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.208+0000 7fc371c9b640 1 -- 172.21.15.31:0/508024877 wait complete. 2024-01-19T04:56:08.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 Processor -- start 2024-01-19T04:56:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 -- start start 2024-01-19T04:56:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc13015d8a0 0x7fc13006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc13015eb70 0x7fc1300cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc1300cca00 0x7fc13015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13006bf30 con 0x7fc13015d8a0 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1300cb9e0 con 0x7fc13015eb70 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.320+0000 7fc1357de640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300cbcc0 con 0x7fc1300cca00 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc13015d8a0 0x7fc13006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc13015d8a0 0x7fc13006b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40122/0 (socket says 172.21.15.31:40122) 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 -- 172.21.15.31:0/3023925059 learned_addr learned my addr 172.21.15.31:0/3023925059 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc127fff640 1 --2- 172.21.15.31:0/3023925059 >> v2:172.21.15.137:3300/0 conn(0x7fc13015eb70 0x7fc1300cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12f7fe640 1 --2- 172.21.15.31:0/3023925059 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc1300cca00 0x7fc13015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 -- 172.21.15.31:0/3023925059 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc1300cca00 msgr2=0x7fc13015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 --2- 172.21.15.31:0/3023925059 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc1300cca00 0x7fc13015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 -- 172.21.15.31:0/3023925059 >> v2:172.21.15.137:3300/0 conn(0x7fc13015eb70 msgr2=0x7fc1300cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 --2- 172.21.15.31:0/3023925059 >> v2:172.21.15.137:3300/0 conn(0x7fc13015eb70 0x7fc1300cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc12effd640 1 -- 172.21.15.31:0/3023925059 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1300cbfa0 con 0x7fc13015d8a0 2024-01-19T04:56:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.321+0000 7fc127fff640 1 --2- 172.21.15.31:0/3023925059 >> v2:172.21.15.137:3300/0 conn(0x7fc13015eb70 0x7fc1300cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc12effd640 1 --2- 172.21.15.31:0/3023925059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc13015d8a0 0x7fc13006b6c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc1300d2220 tx=0x7fc12002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789fc3a59f4e43f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc12e7fc640 1 -- 172.21.15.31:0/3023925059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1200317e0 con 0x7fc13015d8a0 2024-01-19T04:56:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc12e7fc640 1 -- 172.21.15.31:0/3023925059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc120031940 con 0x7fc13015d8a0 2024-01-19T04:56:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc12e7fc640 1 -- 172.21.15.31:0/3023925059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120031cd0 con 0x7fc13015d8a0 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc1357de640 1 -- 172.21.15.31:0/3023925059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc13015d8a0 msgr2=0x7fc13006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc1357de640 1 --2- 172.21.15.31:0/3023925059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc13015d8a0 0x7fc13006b6c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc1300d2220 tx=0x7fc12002b880 comp rx=0 tx=0).stop 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc1357de640 1 -- 172.21.15.31:0/3023925059 shutdown_connections 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc1357de640 1 --2- 172.21.15.31:0/3023925059 >> v2:172.21.15.137:3300/0 conn(0x7fc13015eb70 0x7fc1300cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc1357de640 1 --2- 172.21.15.31:0/3023925059 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc1300cca00 0x7fc13015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc1357de640 1 --2- 172.21.15.31:0/3023925059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc13015d8a0 0x7fc13006b6c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.322+0000 7fc1357de640 1 -- 172.21.15.31:0/3023925059 >> 172.21.15.31:0/3023925059 conn(0x7fc13006abe0 msgr2=0x7fc130066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.323+0000 7fc1357de640 1 -- 172.21.15.31:0/3023925059 shutdown_connections 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.323+0000 7fc1357de640 1 -- 172.21.15.31:0/3023925059 wait complete. 2024-01-19T04:56:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 Processor -- start 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 -- start start 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300cca00 0x7fc1300caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc13015d8a0 0x7fc1300c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc13015eb70 0x7fc1300c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300b5200 con 0x7fc1300cca00 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300b5d80 con 0x7fc13015eb70 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.324+0000 7fc1357de640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300b5450 con 0x7fc13015d8a0 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300cca00 0x7fc1300caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300cca00 0x7fc1300caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40128/0 (socket says 172.21.15.31:40128) 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 -- 172.21.15.31:0/2646803440 learned_addr learned my addr 172.21.15.31:0/2646803440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 -- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc13015d8a0 msgr2=0x7fc1300c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc13015d8a0 0x7fc1300c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 -- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc13015eb70 msgr2=0x7fc1300c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc13015eb70 0x7fc1300c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 -- 172.21.15.31:0/2646803440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1200094d0 con 0x7fc1300cca00 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc12effd640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300cca00 0x7fc1300caea0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc12002b8c0 tx=0x7fc1200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc1277fe640 1 -- 172.21.15.31:0/2646803440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120032900 con 0x7fc1300cca00 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc1277fe640 1 -- 172.21.15.31:0/2646803440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc120032a60 con 0x7fc1300cca00 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1300b5730 con 0x7fc1300cca00 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc1277fe640 1 -- 172.21.15.31:0/2646803440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120032e10 con 0x7fc1300cca00 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.325+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1300cb3e0 con 0x7fc1300cca00 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.326+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1300696e0 con 0x7fc1300cca00 2024-01-19T04:56:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.327+0000 7fc1277fe640 1 -- 172.21.15.31:0/2646803440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc1200453f0 con 0x7fc1300cca00 2024-01-19T04:56:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.327+0000 7fc1277fe640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc11006aed0 0x7fc11006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.327+0000 7fc1277fe640 1 -- 172.21.15.31:0/2646803440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc1200aba00 con 0x7fc1300cca00 2024-01-19T04:56:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.327+0000 7fc127fff640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc11006aed0 0x7fc11006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.328+0000 7fc127fff640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc11006aed0 0x7fc11006d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc13015b5c0 tx=0x7fc118008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.331+0000 7fc1277fe640 1 -- 172.21.15.31:0/2646803440 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc1200768e0 con 0x7fc1300cca00 2024-01-19T04:56:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.510+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1300b4fe0 con 0x7fc1300cca00 2024-01-19T04:56:08.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.511+0000 7fc1277fe640 1 -- 172.21.15.31:0/2646803440 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16720 (secure 0 0 0) 0x7fc120079f00 con 0x7fc1300cca00 2024-01-19T04:56:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:08.511 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:01.578429+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:55:16.957427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-19T04:55:17.841010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.514+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc11006aed0 msgr2=0x7fc11006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.514+0000 7fc1357de640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc11006aed0 0x7fc11006d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc13015b5c0 tx=0x7fc118008040 comp rx=0 tx=0).stop 2024-01-19T04:56:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.514+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300cca00 msgr2=0x7fc1300caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.514+0000 7fc1357de640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300cca00 0x7fc1300caea0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc12002b8c0 tx=0x7fc1200046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 shutdown_connections 2024-01-19T04:56:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc11006aed0 0x7fc11006d390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc13015eb70 0x7fc1300c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc13015d8a0 0x7fc1300c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 --2- 172.21.15.31:0/2646803440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300cca00 0x7fc1300caea0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 >> 172.21.15.31:0/2646803440 conn(0x7fc13006abe0 msgr2=0x7fc1300b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 shutdown_connections 2024-01-19T04:56:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.515+0000 7fc1357de640 1 -- 172.21.15.31:0/2646803440 wait complete. 2024-01-19T04:56:08.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-19T04:56:08.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.632+0000 7f09fb74c640 1 Processor -- start 2024-01-19T04:56:08.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.632+0000 7f09fb74c640 1 -- start start 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09fb74c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f09f40ce090 0x7f09f40d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09fb74c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09f40cd2e0 0x7f09f40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09fb74c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f40cce50 con 0x7f09f415b6b0 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09fb74c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f40d03b0 con 0x7f09f40cd2e0 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09fb74c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f40cc240 con 0x7f09f40ce090 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09f94e8640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f09f415b6b0 0x7f09f40cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58508/0 (socket says 172.21.15.31:58508) 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09f9ce9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09f40cd2e0 0x7f09f40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09f94e8640 1 -- 172.21.15.31:0/2281213109 learned_addr learned my addr 172.21.15.31:0/2281213109 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:08.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.633+0000 7f09f8ce7640 1 --2- 172.21.15.31:0/2281213109 >> v2:172.21.15.137:3300/0 conn(0x7f09f40ce090 0x7f09f40d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.634+0000 7f09f9ce9640 1 -- 172.21.15.31:0/2281213109 >> v2:172.21.15.137:3300/0 conn(0x7f09f40ce090 msgr2=0x7f09f40d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:08.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.634+0000 7f09f9ce9640 1 --2- 172.21.15.31:0/2281213109 >> v2:172.21.15.137:3300/0 conn(0x7f09f40ce090 0x7f09f40d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.634+0000 7f09f9ce9640 1 -- 172.21.15.31:0/2281213109 >> v1:172.21.15.31:6789/0 conn(0x7f09f415b6b0 legacy=0x7f09f40cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.634+0000 7f09f3fff640 1 -- 172.21.15.31:0/2281213109 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996289153 0 0) 0x7f09f40cce50 con 0x7f09f415b6b0 2024-01-19T04:56:08.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.634+0000 7f09f9ce9640 1 -- 172.21.15.31:0/2281213109 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40cc520 con 0x7f09f40cd2e0 2024-01-19T04:56:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.634+0000 7f09f9ce9640 1 --2- 172.21.15.31:0/2281213109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40b3f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f09f40729e0 tx=0x7f09e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4331404da4130a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.634+0000 7f09f3fff640 1 -- 172.21.15.31:0/2281213109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e402cc80 con 0x7f09f40cd2e0 2024-01-19T04:56:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09f3fff640 1 -- 172.21.15.31:0/2281213109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e402cde0 con 0x7f09f40cd2e0 2024-01-19T04:56:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09f3fff640 1 -- 172.21.15.31:0/2281213109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4034660 con 0x7f09f40cd2e0 2024-01-19T04:56:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 -- 172.21.15.31:0/2281213109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 msgr2=0x7f09f40b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 --2- 172.21.15.31:0/2281213109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40b3f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f09f40729e0 tx=0x7f09e402bed0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 -- 172.21.15.31:0/2281213109 shutdown_connections 2024-01-19T04:56:08.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 --2- 172.21.15.31:0/2281213109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40b3f50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 --2- 172.21.15.31:0/2281213109 >> v2:172.21.15.137:3300/0 conn(0x7f09f40ce090 0x7f09f40d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 -- 172.21.15.31:0/2281213109 >> 172.21.15.31:0/2281213109 conn(0x7f09f40d1100 msgr2=0x7f09f40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:08.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 -- 172.21.15.31:0/2281213109 shutdown_connections 2024-01-19T04:56:08.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.635+0000 7f09fb74c640 1 -- 172.21.15.31:0/2281213109 wait complete. 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.636+0000 7f09fb74c640 1 Processor -- start 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.636+0000 7f09fb74c640 1 -- start start 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09fb74c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09fb74c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09f40ce090 0x7f09f414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09fb74c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f09f415b6b0 0x7f09f406f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09fb74c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f414eb40 con 0x7f09f40cd2e0 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09fb74c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40c81d0 con 0x7f09f40ce090 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09fb74c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40c8420 con 0x7f09f415b6b0 2024-01-19T04:56:08.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40146/0 (socket says 172.21.15.31:40146) 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 -- 172.21.15.31:0/3624622806 learned_addr learned my addr 172.21.15.31:0/3624622806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f8ce7640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09f40ce090 0x7f09f414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f9ce9640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f09f415b6b0 0x7f09f406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 -- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f09f415b6b0 msgr2=0x7f09f406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f09f415b6b0 0x7f09f406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 -- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09f40ce090 msgr2=0x7f09f414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09f40ce090 0x7f09f414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.637+0000 7f09f94e8640 1 -- 172.21.15.31:0/3624622806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4009a70 con 0x7f09f40cd2e0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.638+0000 7f09f94e8640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40c94c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f09ec001eb0 tx=0x7f09ec00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.638+0000 7f09f1ffb640 1 -- 172.21.15.31:0/3624622806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0074e0 con 0x7f09f40cd2e0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.638+0000 7f09f1ffb640 1 -- 172.21.15.31:0/3624622806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec007640 con 0x7f09f40cd2e0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.638+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f40c8700 con 0x7f09f40cd2e0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.638+0000 7f09f1ffb640 1 -- 172.21.15.31:0/3624622806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0059e0 con 0x7f09f40cd2e0 2024-01-19T04:56:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.638+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f40c8cc0 con 0x7f09f40cd2e0 2024-01-19T04:56:08.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.639+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f4074900 con 0x7f09f40cd2e0 2024-01-19T04:56:08.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.639+0000 7f09f1ffb640 1 -- 172.21.15.31:0/3624622806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f09ec005b40 con 0x7f09f40cd2e0 2024-01-19T04:56:08.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.640+0000 7f09f1ffb640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f09e006aed0 0x7f09e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.640+0000 7f09f1ffb640 1 -- 172.21.15.31:0/3624622806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f09ec088c20 con 0x7f09f40cd2e0 2024-01-19T04:56:08.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.640+0000 7f09f8ce7640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f09e006aed0 0x7f09e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.640+0000 7f09f8ce7640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f09e006aed0 0x7f09e006d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09f40cb5e0 tx=0x7f09e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.644+0000 7f09f1ffb640 1 -- 172.21.15.31:0/3624622806 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f09ec053bc0 con 0x7f09f40cd2e0 2024-01-19T04:56:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.860+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f09f4067030 con 0x7f09f40cd2e0 2024-01-19T04:56:08.860 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-19T04:56:08.860+0000 7f70c6e70640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-19T04:56:08.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.862+0000 7f09f1ffb640 1 -- 172.21.15.31:0/3624622806 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f09ec017020 con 0x7f09f40cd2e0 2024-01-19T04:56:08.861 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f09e006aed0 msgr2=0x7f09e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f09e006aed0 0x7f09e006d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09f40cb5e0 tx=0x7f09e8008040 comp rx=0 tx=0).stop 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 msgr2=0x7f09f40c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40c94c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f09ec001eb0 tx=0x7f09ec00c890 comp rx=0 tx=0).stop 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 shutdown_connections 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f09e006aed0 0x7f09e006d390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f09f415b6b0 0x7f09f406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09f40ce090 0x7f09f414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 --2- 172.21.15.31:0/3624622806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f40cd2e0 0x7f09f40c94c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 >> 172.21.15.31:0/3624622806 conn(0x7f09f40d1100 msgr2=0x7f09f40d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 shutdown_connections 2024-01-19T04:56:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.865+0000 7f09fb74c640 1 -- 172.21.15.31:0/3624622806 wait complete. 2024-01-19T04:56:08.866 INFO:tasks.ceph.mds.a.smithi031.stderr: -2855> 2024-01-19T04:55:23.495+0000 7f14d06fd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-19T04:56:08.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-19T04:56:08.895 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-01-19T04:56:08.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.983+0000 7f5b5f63a640 1 Processor -- start 2024-01-19T04:56:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.983+0000 7f5b5f63a640 1 -- start start 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5f63a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5b5815d700 0x7f5b580d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5f63a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b580d1fe0 0x7f5b580b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5f63a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b580cef10 con 0x7f5b580d11b0 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5f63a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580ce300 con 0x7f5b580d1fe0 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5f63a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580ce5e0 con 0x7f5b5815d700 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5dbd7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b580d1fe0 0x7f5b580b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5dbd7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b580d1fe0 0x7f5b580b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40180/0 (socket says 172.21.15.31:40180) 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5dbd7640 1 -- 172.21.15.31:0/2326516412 learned_addr learned my addr 172.21.15.31:0/2326516412 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5d3d6640 1 --2- 172.21.15.31:0/2326516412 >> v2:172.21.15.137:3300/0 conn(0x7f5b5815d700 0x7f5b580d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5dbd7640 1 -- 172.21.15.31:0/2326516412 >> v2:172.21.15.137:3300/0 conn(0x7f5b5815d700 msgr2=0x7f5b580d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5dbd7640 1 --2- 172.21.15.31:0/2326516412 >> v2:172.21.15.137:3300/0 conn(0x7f5b5815d700 0x7f5b580d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5dbd7640 1 -- 172.21.15.31:0/2326516412 >> v1:172.21.15.137:6789/0 conn(0x7f5b580d11b0 legacy=0x7f5b580d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5dbd7640 1 -- 172.21.15.31:0/2326516412 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580ce8c0 con 0x7f5b580d1fe0 2024-01-19T04:56:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.984+0000 7f5b5d3d6640 1 --2- 172.21.15.31:0/2326516412 >> v2:172.21.15.137:3300/0 conn(0x7f5b5815d700 0x7f5b580d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.985+0000 7f5b5dbd7640 1 --2- 172.21.15.31:0/2326516412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d1fe0 0x7f5b580b2890 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5b580b5e70 tx=0x7f5b5002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3ef7c71902278d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.985+0000 7f5b4ffff640 1 -- 172.21.15.31:0/2326516412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5002cb20 con 0x7f5b580d1fe0 2024-01-19T04:56:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.985+0000 7f5b4ffff640 1 -- 172.21.15.31:0/2326516412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5002cc80 con 0x7f5b580d1fe0 2024-01-19T04:56:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.985+0000 7f5b4ffff640 1 -- 172.21.15.31:0/2326516412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b500343f0 con 0x7f5b580d1fe0 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.985+0000 7f5b5f63a640 1 -- 172.21.15.31:0/2326516412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d1fe0 msgr2=0x7f5b580b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.985+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/2326516412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d1fe0 0x7f5b580b2890 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5b580b5e70 tx=0x7f5b5002bf80 comp rx=0 tx=0).stop 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.986+0000 7f5b5f63a640 1 -- 172.21.15.31:0/2326516412 shutdown_connections 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.986+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/2326516412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d1fe0 0x7f5b580b2890 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.986+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/2326516412 >> v2:172.21.15.137:3300/0 conn(0x7f5b5815d700 0x7f5b580d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.986+0000 7f5b5f63a640 1 -- 172.21.15.31:0/2326516412 >> 172.21.15.31:0/2326516412 conn(0x7f5b580d32f0 msgr2=0x7f5b580d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.986+0000 7f5b5f63a640 1 -- 172.21.15.31:0/2326516412 shutdown_connections 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.986+0000 7f5b5f63a640 1 -- 172.21.15.31:0/2326516412 wait complete. 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.987+0000 7f5b5f63a640 1 Processor -- start 2024-01-19T04:56:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.987+0000 7f5b5f63a640 1 -- start start 2024-01-19T04:56:08.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.987+0000 7f5b5f63a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d11b0 0x7f5b581f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.987+0000 7f5b5f63a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b580d1fe0 0x7f5b580b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.987+0000 7f5b5f63a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b5815d700 0x7f5b580b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.987+0000 7f5b5f63a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581f4ef0 con 0x7f5b580d11b0 2024-01-19T04:56:08.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.987+0000 7f5b5f63a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581f85f0 con 0x7f5b5815d700 2024-01-19T04:56:08.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5f63a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580c7f10 con 0x7f5b580d1fe0 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d11b0 0x7f5b581f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d11b0 0x7f5b581f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40188/0 (socket says 172.21.15.31:40188) 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 -- 172.21.15.31:0/1700926060 learned_addr learned my addr 172.21.15.31:0/1700926060 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5dbd7640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b5815d700 0x7f5b580b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5cbd5640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b580d1fe0 0x7f5b580b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 -- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b580d1fe0 msgr2=0x7f5b580b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b580d1fe0 0x7f5b580b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 -- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b5815d700 msgr2=0x7f5b580b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b5815d700 0x7f5b580b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 -- 172.21.15.31:0/1700926060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b50009b30 con 0x7f5b580d11b0 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5d3d6640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d11b0 0x7f5b581f49b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400c7f0 tx=0x7f5b5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b4dffb640 1 -- 172.21.15.31:0/1700926060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54007820 con 0x7f5b580d11b0 2024-01-19T04:56:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b4dffb640 1 -- 172.21.15.31:0/1700926060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b54007980 con 0x7f5b580d11b0 2024-01-19T04:56:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b580b8d30 con 0x7f5b580d11b0 2024-01-19T04:56:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b4dffb640 1 -- 172.21.15.31:0/1700926060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54007cf0 con 0x7f5b580d11b0 2024-01-19T04:56:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.988+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b580c8160 con 0x7f5b580d11b0 2024-01-19T04:56:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.989+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b58074900 con 0x7f5b580d11b0 2024-01-19T04:56:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.990+0000 7f5b4dffb640 1 -- 172.21.15.31:0/1700926060 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5b540113f0 con 0x7f5b580d11b0 2024-01-19T04:56:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.990+0000 7f5b4dffb640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5b3c06aed0 0x7f5b3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.990+0000 7f5b4dffb640 1 -- 172.21.15.31:0/1700926060 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f5b54089aa0 con 0x7f5b580d11b0 2024-01-19T04:56:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.991+0000 7f5b5cbd5640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5b3c06aed0 0x7f5b3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.991+0000 7f5b5cbd5640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5b3c06aed0 0x7f5b3c06d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b5815c210 tx=0x7f5b48009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:08.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:08.995+0000 7f5b4dffb640 1 -- 172.21.15.31:0/1700926060 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5b54054800 con 0x7f5b580d11b0 2024-01-19T04:56:09.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.211+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5b580d16b0 con 0x7f5b580d11b0 2024-01-19T04:56:09.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.864+0000 7f5b4dffb640 1 -- 172.21.15.31:0/1700926060 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5b54057e20 con 0x7f5b580d11b0 2024-01-19T04:56:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5b3c06aed0 msgr2=0x7f5b3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5b3c06aed0 0x7f5b3c06d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b5815c210 tx=0x7f5b48009040 comp rx=0 tx=0).stop 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d11b0 msgr2=0x7f5b581f49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d11b0 0x7f5b581f49b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400c7f0 tx=0x7f5b5400ccb0 comp rx=0 tx=0).stop 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 shutdown_connections 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5b3c06aed0 0x7f5b3c06d390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b5815d700 0x7f5b580b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b580d1fe0 0x7f5b580b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 --2- 172.21.15.31:0/1700926060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b580d11b0 0x7f5b581f49b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 >> 172.21.15.31:0/1700926060 conn(0x7f5b580d32f0 msgr2=0x7f5b5806b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 shutdown_connections 2024-01-19T04:56:09.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:09.867+0000 7f5b5f63a640 1 -- 172.21.15.31:0/1700926060 wait complete. 2024-01-19T04:56:09.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-19T04:56:10.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.034+0000 7fe7c1cb4640 1 Processor -- start 2024-01-19T04:56:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.034+0000 7fe7c1cb4640 1 -- start start 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7c1cb4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7bc06acc0 0x7fe7bc15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7c1cb4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc06ebe0 con 0x7fe7bc071f20 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7c1cb4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0723b0 con 0x7fe7bc144a70 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7c1cb4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc15fe60 con 0x7fe7bc06acc0 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7baffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7bc06acc0 0x7fe7bc15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7bbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe7bc071f20 0x7fe7bc06f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33586/0 (socket says 172.21.15.31:33586) 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7bbfff640 1 -- 172.21.15.31:0/11679252 learned_addr learned my addr 172.21.15.31:0/11679252 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7baffd640 1 --2- 172.21.15.31:0/11679252 >> v2:172.21.15.31:3300/0 conn(0x7fe7bc06acc0 0x7fe7bc15ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40060/0 (socket says 172.21.15.31:40060) 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7baffd640 1 -- 172.21.15.31:0/11679252 >> v1:172.21.15.137:6789/0 conn(0x7fe7bc144a70 legacy=0x7fe7bc06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7baffd640 1 -- 172.21.15.31:0/11679252 >> v1:172.21.15.31:6789/0 conn(0x7fe7bc071f20 legacy=0x7fe7bc06f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.035+0000 7fe7baffd640 1 -- 172.21.15.31:0/11679252 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc15f5c0 con 0x7fe7bc06acc0 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.036+0000 7fe7baffd640 1 --2- 172.21.15.31:0/11679252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc06acc0 0x7fe7bc15ca20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00097a0 tx=0x7fe7b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4e1ec3b4539db4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.036+0000 7fe7ba7fc640 1 -- 172.21.15.31:0/11679252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0031830 con 0x7fe7bc06acc0 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.036+0000 7fe7ba7fc640 1 -- 172.21.15.31:0/11679252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0031990 con 0x7fe7bc06acc0 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.036+0000 7fe7ba7fc640 1 -- 172.21.15.31:0/11679252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0031d20 con 0x7fe7bc06acc0 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.037+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/11679252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc06acc0 msgr2=0x7fe7bc15ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.037+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/11679252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc06acc0 0x7fe7bc15ca20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00097a0 tx=0x7fe7b002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.037+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/11679252 shutdown_connections 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.037+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/11679252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc06acc0 0x7fe7bc15ca20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.037+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/11679252 >> 172.21.15.31:0/11679252 conn(0x7fe7bc06d750 msgr2=0x7fe7bc06db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.037+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/11679252 shutdown_connections 2024-01-19T04:56:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.037+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/11679252 wait complete. 2024-01-19T04:56:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.038+0000 7fe7c1cb4640 1 Processor -- start 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.038+0000 7fe7c1cb4640 1 -- start start 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7c1cb4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7bc06acc0 0x7fe7bc13f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7c1cb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc071f20 0x7fe7bc0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7c1cb4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7bc144a70 0x7fe7bc15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7c1cb4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0c98d0 con 0x7fe7bc071f20 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7c1cb4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0c9000 con 0x7fe7bc06acc0 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7c1cb4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0cc020 con 0x7fe7bc144a70 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc071f20 0x7fe7bc0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc071f20 0x7fe7bc0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40076/0 (socket says 172.21.15.31:40076) 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 -- 172.21.15.31:0/2608874097 learned_addr learned my addr 172.21.15.31:0/2608874097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 -- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7bc144a70 msgr2=0x7fe7bc15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7bbfff640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7bc144a70 0x7fe7bc15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7bb7fe640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7bc06acc0 0x7fe7bc13f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7bc144a70 0x7fe7bc15dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 -- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7bc06acc0 msgr2=0x7fe7bc13f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7bc06acc0 0x7fe7bc13f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.039+0000 7fe7baffd640 1 -- 172.21.15.31:0/2608874097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b0009400 con 0x7fe7bc071f20 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.040+0000 7fe7baffd640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc071f20 0x7fe7bc0c7570 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002bee0 tx=0x7fe7b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.040+0000 7fe7a3fff640 1 -- 172.21.15.31:0/2608874097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b003ba70 con 0x7fe7bc071f20 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.040+0000 7fe7a3fff640 1 -- 172.21.15.31:0/2608874097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b003bbf0 con 0x7fe7bc071f20 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.040+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc0c62e0 con 0x7fe7bc071f20 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.040+0000 7fe7a3fff640 1 -- 172.21.15.31:0/2608874097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00453f0 con 0x7fe7bc071f20 2024-01-19T04:56:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.040+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc0c9210 con 0x7fe7bc071f20 2024-01-19T04:56:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.041+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc0744b0 con 0x7fe7bc071f20 2024-01-19T04:56:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.041+0000 7fe7a3fff640 1 -- 172.21.15.31:0/2608874097 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe7b004e450 con 0x7fe7bc071f20 2024-01-19T04:56:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.042+0000 7fe7a3fff640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe7a806aed0 0x7fe7a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.042+0000 7fe7a3fff640 1 -- 172.21.15.31:0/2608874097 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fe7b00ab940 con 0x7fe7bc071f20 2024-01-19T04:56:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.042+0000 7fe7bb7fe640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe7a806aed0 0x7fe7a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.042+0000 7fe7bb7fe640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe7a806aed0 0x7fe7a806d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac001500 tx=0x7fe7ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:10.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.046+0000 7fe7a3fff640 1 -- 172.21.15.31:0/2608874097 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe7b0076740 con 0x7fe7bc071f20 2024-01-19T04:56:10.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.225+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7bc0696e0 con 0x7fe7bc071f20 2024-01-19T04:56:10.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.865+0000 7fe7a3fff640 1 -- 172.21.15.31:0/2608874097 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe7b0035020 con 0x7fe7bc071f20 2024-01-19T04:56:10.865 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-01-19T04:56:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe7a806aed0 msgr2=0x7fe7a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe7a806aed0 0x7fe7a806d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac001500 tx=0x7fe7ac009040 comp rx=0 tx=0).stop 2024-01-19T04:56:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc071f20 msgr2=0x7fe7bc0c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc071f20 0x7fe7bc0c7570 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002bee0 tx=0x7fe7b00046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 shutdown_connections 2024-01-19T04:56:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe7a806aed0 0x7fe7a806d390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7bc144a70 0x7fe7bc15dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc071f20 0x7fe7bc0c7570 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 --2- 172.21.15.31:0/2608874097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7bc06acc0 0x7fe7bc13f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.868+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 >> 172.21.15.31:0/2608874097 conn(0x7fe7bc06d750 msgr2=0x7fe7bc06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.869+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 shutdown_connections 2024-01-19T04:56:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.869+0000 7fe7c1cb4640 1 -- 172.21.15.31:0/2608874097 wait complete. 2024-01-19T04:56:10.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-19T04:56:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.994+0000 7f3457aaf640 1 Processor -- start 2024-01-19T04:56:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.994+0000 7f3457aaf640 1 -- start start 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f3457aaf640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345009a160 0x7f34500cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f3457aaf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f345006c940 0x7f34500cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f3457aaf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345006bb40 con 0x7f345006c560 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f3457aaf640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34500cfeb0 con 0x7f345006c940 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f3457aaf640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34500d00a0 con 0x7f345009a160 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345604c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f345006c940 0x7f34500cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345604c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f345006c940 0x7f34500cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40078/0 (socket says 172.21.15.31:40078) 2024-01-19T04:56:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345604c640 1 -- 172.21.15.31:0/2867235115 learned_addr learned my addr 172.21.15.31:0/2867235115 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345584b640 1 --1- 172.21.15.31:0/2867235115 >> v1:172.21.15.31:6789/0 conn(0x7f345006c560 0x7f3450099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33598/0 (socket says 172.21.15.31:33598) 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345604c640 1 -- 172.21.15.31:0/2867235115 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345009a160 msgr2=0x7f34500cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345504a640 1 --2- 172.21.15.31:0/2867235115 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345009a160 0x7f34500cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f3454849640 1 -- 172.21.15.31:0/2867235115 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691548546 0 0) 0x7f345006bb40 con 0x7f345006c560 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345604c640 1 --2- 172.21.15.31:0/2867235115 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345009a160 0x7f34500cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345604c640 1 -- 172.21.15.31:0/2867235115 >> v1:172.21.15.31:6789/0 conn(0x7f345006c560 legacy=0x7f3450099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345604c640 1 -- 172.21.15.31:0/2867235115 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500d0380 con 0x7f345006c940 2024-01-19T04:56:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.995+0000 7f345504a640 1 --2- 172.21.15.31:0/2867235115 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345009a160 0x7f34500cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.996+0000 7f345604c640 1 --2- 172.21.15.31:0/2867235115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345006c940 0x7f34500cf970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3448009e50 tx=0x7f344802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc91372d513a10dc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.996+0000 7f3454849640 1 -- 172.21.15.31:0/2867235115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344802cb00 con 0x7f345006c940 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.996+0000 7f3454849640 1 -- 172.21.15.31:0/2867235115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344802cc60 con 0x7f345006c940 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.996+0000 7f3454849640 1 -- 172.21.15.31:0/2867235115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480343f0 con 0x7f345006c940 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 -- 172.21.15.31:0/2867235115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345006c940 msgr2=0x7f34500cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 --2- 172.21.15.31:0/2867235115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345006c940 0x7f34500cf970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3448009e50 tx=0x7f344802bf60 comp rx=0 tx=0).stop 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 -- 172.21.15.31:0/2867235115 shutdown_connections 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 --2- 172.21.15.31:0/2867235115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345006c940 0x7f34500cf970 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 --2- 172.21.15.31:0/2867235115 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345009a160 0x7f34500cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 -- 172.21.15.31:0/2867235115 >> 172.21.15.31:0/2867235115 conn(0x7f34500b2bb0 msgr2=0x7f34500b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 -- 172.21.15.31:0/2867235115 shutdown_connections 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.997+0000 7f3457aaf640 1 -- 172.21.15.31:0/2867235115 wait complete. 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.998+0000 7f3457aaf640 1 Processor -- start 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.998+0000 7f3457aaf640 1 -- start start 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f3457aaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f345006c560 0x7f34501f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f3457aaf640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345006c940 0x7f34500c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f3457aaf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345009a160 0x7f345015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f3457aaf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500c6420 con 0x7f345009a160 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f3457aaf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500c5560 con 0x7f345006c560 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f3457aaf640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34500c5840 con 0x7f345006c940 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345009a160 0x7f345015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345009a160 0x7f345015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40092/0 (socket says 172.21.15.31:40092) 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 -- 172.21.15.31:0/71102959 learned_addr learned my addr 172.21.15.31:0/71102959 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345504a640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345006c940 0x7f34500c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345584b640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f345006c560 0x7f34501f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 -- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345006c940 msgr2=0x7f34500c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345006c940 0x7f34500c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 -- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f345006c560 msgr2=0x7f34501f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f345006c560 0x7f34501f07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 -- 172.21.15.31:0/71102959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009b30 con 0x7f345009a160 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345504a640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345006c940 0x7f34500c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345584b640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f345006c560 0x7f34501f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f345604c640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345009a160 0x7f345015b7f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3448036040 tx=0x7f3448004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:10.999+0000 7f34467fc640 1 -- 172.21.15.31:0/71102959 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344803e450 con 0x7f345009a160 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.000+0000 7f34467fc640 1 -- 172.21.15.31:0/71102959 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344803e5d0 con 0x7f345009a160 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.000+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34500c5b20 con 0x7f345009a160 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.000+0000 7f34467fc640 1 -- 172.21.15.31:0/71102959 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344803e780 con 0x7f345009a160 2024-01-19T04:56:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.000+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34500c5130 con 0x7f345009a160 2024-01-19T04:56:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.001+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34500696e0 con 0x7f345009a160 2024-01-19T04:56:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.001+0000 7f34467fc640 1 -- 172.21.15.31:0/71102959 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f344803e8e0 con 0x7f345009a160 2024-01-19T04:56:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.001+0000 7f34467fc640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406aed0 0x7f343406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:11.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.001+0000 7f34467fc640 1 -- 172.21.15.31:0/71102959 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f34480ac0c0 con 0x7f345009a160 2024-01-19T04:56:11.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.002+0000 7f345584b640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406aed0 0x7f343406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:11.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.002+0000 7f345584b640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406aed0 0x7f343406d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f344c000f60 tx=0x7f344c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.006+0000 7f34467fc640 1 -- 172.21.15.31:0/71102959 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3448077060 con 0x7f345009a160 2024-01-19T04:56:11.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.189+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34500b3a70 con 0x7f345009a160 2024-01-19T04:56:11.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.870+0000 7f34467fc640 1 -- 172.21.15.31:0/71102959 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f344807a680 con 0x7f345009a160 2024-01-19T04:56:11.870 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-01-19T04:56:11.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406aed0 msgr2=0x7f343406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406aed0 0x7f343406d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f344c000f60 tx=0x7f344c009040 comp rx=0 tx=0).stop 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345009a160 msgr2=0x7f345015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345009a160 0x7f345015b7f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3448036040 tx=0x7f3448004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 shutdown_connections 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f343406aed0 0x7f343406d390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345009a160 0x7f345015b7f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f345006c940 0x7f34500c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 --2- 172.21.15.31:0/71102959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f345006c560 0x7f34501f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 >> 172.21.15.31:0/71102959 conn(0x7f34500b2bb0 msgr2=0x7f3450099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 shutdown_connections 2024-01-19T04:56:11.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:11.875+0000 7f3457aaf640 1 -- 172.21.15.31:0/71102959 wait complete. 2024-01-19T04:56:11.890 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-19T04:56:11.890 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-19T04:56:11.890 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-19T04:56:11.890 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:11.890 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-19T04:56:11.890 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-19T04:56:11.891 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-19T04:56:11.891 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:11.891 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-19T04:56:11.891 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-19T04:56:11.891 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-19T04:56:11.891 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:11.892 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-19T04:56:11.892 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-19T04:56:11.892 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-19T04:56:11.892 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:11.902 INFO:tasks.ceph.mds.c:Stopped 2024-01-19T04:56:11.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-19T04:56:11.907 INFO:tasks.ceph.mds.b:Stopped 2024-01-19T04:56:11.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-19T04:56:11.907 INFO:tasks.ceph.mds.d:Stopped 2024-01-19T04:56:11.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-19T04:56:11.945 INFO:tasks.ceph.mds.c:Started 2024-01-19T04:56:11.945 DEBUG:teuthology.parallel:result is None 2024-01-19T04:56:11.949 INFO:tasks.ceph.mds.b:Started 2024-01-19T04:56:11.950 DEBUG:teuthology.parallel:result is None 2024-01-19T04:56:11.950 INFO:tasks.ceph.mds.d:Started 2024-01-19T04:56:11.950 DEBUG:teuthology.parallel:result is None 2024-01-19T04:56:12.027 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2024-01-19T04:56:12.032 INFO:tasks.ceph.mds.b.smithi137.stdout:starting mds.b at 2024-01-19T04:56:12.033 INFO:tasks.ceph.mds.d.smithi137.stdout:starting mds.d at 2024-01-19T04:56:13.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:56:13.639 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:56:15.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-19T04:56:17.903 INFO:tasks.ceph.mds.a:Stopped 2024-01-19T04:56:17.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-19T04:56:17.906 INFO:tasks.ceph.mds.a:Started 2024-01-19T04:56:17.906 DEBUG:teuthology.parallel:result is None 2024-01-19T04:56:17.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-19T04:56:17.992 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-01-19T04:56:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.022+0000 7fc294bec640 1 Processor -- start 2024-01-19T04:56:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.022+0000 7fc294bec640 1 -- start start 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc294bec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2900d1c20 0x7fc2900d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc294bec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d3ea0 0x7fc2900b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc294bec640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2900cf160 con 0x7fc29015d720 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc294bec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2900ce8c0 con 0x7fc2900d1c20 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc294bec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900ceba0 con 0x7fc2900d3ea0 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc285d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2900d1c20 0x7fc2900d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc285d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2900d1c20 0x7fc2900d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40118/0 (socket says 172.21.15.31:40118) 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc285d9b640 1 -- 172.21.15.31:0/4014191681 learned_addr learned my addr 172.21.15.31:0/4014191681 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc28ed9d640 1 --2- 172.21.15.31:0/4014191681 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d3ea0 0x7fc2900b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc285d9b640 1 -- 172.21.15.31:0/4014191681 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d3ea0 msgr2=0x7fc2900b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc285d9b640 1 --2- 172.21.15.31:0/4014191681 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d3ea0 0x7fc2900b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc285d9b640 1 -- 172.21.15.31:0/4014191681 >> v1:172.21.15.137:6789/0 conn(0x7fc29015d720 legacy=0x7fc2900d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc285d9b640 1 -- 172.21.15.31:0/4014191681 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2900cee80 con 0x7fc2900d1c20 2024-01-19T04:56:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.023+0000 7fc28ed9d640 1 --2- 172.21.15.31:0/4014191681 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d3ea0 0x7fc2900b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.024+0000 7fc285d9b640 1 --2- 172.21.15.31:0/4014191681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2900d1c20 0x7fc2900d0ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc278009870 tx=0x7fc27802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3e7dc9c46f296d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.024+0000 7fc28dd9b640 1 -- 172.21.15.31:0/4014191681 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278031740 con 0x7fc2900d1c20 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.024+0000 7fc28dd9b640 1 -- 172.21.15.31:0/4014191681 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780318a0 con 0x7fc2900d1c20 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.024+0000 7fc28dd9b640 1 -- 172.21.15.31:0/4014191681 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278031c30 con 0x7fc2900d1c20 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 -- 172.21.15.31:0/4014191681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2900d1c20 msgr2=0x7fc2900d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 --2- 172.21.15.31:0/4014191681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2900d1c20 0x7fc2900d0ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc278009870 tx=0x7fc27802b7e0 comp rx=0 tx=0).stop 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 -- 172.21.15.31:0/4014191681 shutdown_connections 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 --2- 172.21.15.31:0/4014191681 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d3ea0 0x7fc2900b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 --2- 172.21.15.31:0/4014191681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2900d1c20 0x7fc2900d0ee0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 -- 172.21.15.31:0/4014191681 >> 172.21.15.31:0/4014191681 conn(0x7fc2900d3310 msgr2=0x7fc2900d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 -- 172.21.15.31:0/4014191681 shutdown_connections 2024-01-19T04:56:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.025+0000 7fc294bec640 1 -- 172.21.15.31:0/4014191681 wait complete. 2024-01-19T04:56:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.026+0000 7fc294bec640 1 Processor -- start 2024-01-19T04:56:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.026+0000 7fc294bec640 1 -- start start 2024-01-19T04:56:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc294bec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d1c20 0x7fc2900cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc294bec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2900d3ea0 0x7fc2900a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc294bec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29015d720 0x7fc2900b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc294bec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900cb650 con 0x7fc29015d720 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc294bec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900ca3e0 con 0x7fc2900d3ea0 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc294bec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900c7c90 con 0x7fc2900d1c20 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc28ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29015d720 0x7fc2900b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc28ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29015d720 0x7fc2900b80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40130/0 (socket says 172.21.15.31:40130) 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc28ed9d640 1 -- 172.21.15.31:0/2545605180 learned_addr learned my addr 172.21.15.31:0/2545605180 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.027+0000 7fc28e59c640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d1c20 0x7fc2900cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc285d9b640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2900d3ea0 0x7fc2900a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc28ed9d640 1 -- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d1c20 msgr2=0x7fc2900cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc28ed9d640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d1c20 0x7fc2900cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc28ed9d640 1 -- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2900d3ea0 msgr2=0x7fc2900a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc28ed9d640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2900d3ea0 0x7fc2900a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc28ed9d640 1 -- 172.21.15.31:0/2545605180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2780094d0 con 0x7fc29015d720 2024-01-19T04:56:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc285d9b640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2900d3ea0 0x7fc2900a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc28ed9d640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29015d720 0x7fc2900b80e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc28800ced0 tx=0x7fc28800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc2877fe640 1 -- 172.21.15.31:0/2545605180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288003b90 con 0x7fc29015d720 2024-01-19T04:56:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc2877fe640 1 -- 172.21.15.31:0/2545605180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc288003d10 con 0x7fc29015d720 2024-01-19T04:56:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.028+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2900ca630 con 0x7fc29015d720 2024-01-19T04:56:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.029+0000 7fc2877fe640 1 -- 172.21.15.31:0/2545605180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288004260 con 0x7fc29015d720 2024-01-19T04:56:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.029+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2900cabf0 con 0x7fc29015d720 2024-01-19T04:56:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.029+0000 7fc2877fe640 1 -- 172.21.15.31:0/2545605180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc2880043c0 con 0x7fc29015d720 2024-01-19T04:56:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.030+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc290074900 con 0x7fc29015d720 2024-01-19T04:56:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.030+0000 7fc2877fe640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc27006aed0 0x7fc27006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.030+0000 7fc2877fe640 1 -- 172.21.15.31:0/2545605180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc28808bd60 con 0x7fc29015d720 2024-01-19T04:56:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.030+0000 7fc28e59c640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc27006aed0 0x7fc27006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.031+0000 7fc28e59c640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc27006aed0 0x7fc27006d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc280006fd0 tx=0x7fc280008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.035+0000 7fc2877fe640 1 -- 172.21.15.31:0/2545605180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc28808f060 con 0x7fc29015d720 2024-01-19T04:56:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:18.226+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc2900d17b0 con 0x7fc29015d720 2024-01-19T04:56:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.892+0000 7fc2877fe640 1 -- 172.21.15.31:0/2545605180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc288054d30 con 0x7fc29015d720 2024-01-19T04:56:19.892 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc27006aed0 msgr2=0x7fc27006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc27006aed0 0x7fc27006d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc280006fd0 tx=0x7fc280008040 comp rx=0 tx=0).stop 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29015d720 msgr2=0x7fc2900b80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29015d720 0x7fc2900b80e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc28800ced0 tx=0x7fc28800a710 comp rx=0 tx=0).stop 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 shutdown_connections 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc27006aed0 0x7fc27006d390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29015d720 0x7fc2900b80e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2900d3ea0 0x7fc2900a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 --2- 172.21.15.31:0/2545605180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2900d1c20 0x7fc2900cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 >> 172.21.15.31:0/2545605180 conn(0x7fc2900d3310 msgr2=0x7fc29006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 shutdown_connections 2024-01-19T04:56:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:19.895+0000 7fc294bec640 1 -- 172.21.15.31:0/2545605180 wait complete. 2024-01-19T04:56:19.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-19T04:56:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf32f26640 1 Processor -- start 2024-01-19T04:56:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf32f26640 1 -- start start 2024-01-19T04:56:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf32f26640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c0d2e50 0x7fbf2c0b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf32f26640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf2c0b4d00 0x7fbf2c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf32f26640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2c160df0 con 0x7fbf2c160a10 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf32f26640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2c0b56b0 con 0x7fbf2c0b4d00 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf32f26640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0b7300 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.016+0000 7fbf314c3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf2c0b4d00 0x7fbf2c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf314c3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf2c0b4d00 0x7fbf2c0b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57920/0 (socket says 172.21.15.31:57920) 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf314c3640 1 -- 172.21.15.31:0/1767232788 learned_addr learned my addr 172.21.15.31:0/1767232788 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf2bfff640 1 --2- 172.21.15.31:0/1767232788 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c0d2e50 0x7fbf2c0b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf314c3640 1 -- 172.21.15.31:0/1767232788 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c0d2e50 msgr2=0x7fbf2c0b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf314c3640 1 --2- 172.21.15.31:0/1767232788 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c0d2e50 0x7fbf2c0b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf314c3640 1 -- 172.21.15.31:0/1767232788 >> v1:172.21.15.137:6789/0 conn(0x7fbf2c160a10 legacy=0x7fbf2c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf314c3640 1 -- 172.21.15.31:0/1767232788 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0d1db0 con 0x7fbf2c0b4d00 2024-01-19T04:56:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.017+0000 7fbf2bfff640 1 --2- 172.21.15.31:0/1767232788 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c0d2e50 0x7fbf2c0b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.018+0000 7fbf314c3640 1 --2- 172.21.15.31:0/1767232788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0b4d00 0x7fbf2c0b5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009e50 tx=0x7fbf2402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ad1b3e66fd3b83 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.018+0000 7fbf2b7fe640 1 -- 172.21.15.31:0/1767232788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2402cb90 con 0x7fbf2c0b4d00 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.018+0000 7fbf2b7fe640 1 -- 172.21.15.31:0/1767232788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2402ccf0 con 0x7fbf2c0b4d00 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.018+0000 7fbf2b7fe640 1 -- 172.21.15.31:0/1767232788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24034610 con 0x7fbf2c0b4d00 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.018+0000 7fbf32f26640 1 -- 172.21.15.31:0/1767232788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0b4d00 msgr2=0x7fbf2c0b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.018+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1767232788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0b4d00 0x7fbf2c0b5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009e50 tx=0x7fbf2402bff0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.019+0000 7fbf32f26640 1 -- 172.21.15.31:0/1767232788 shutdown_connections 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.019+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1767232788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0b4d00 0x7fbf2c0b5170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.019+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1767232788 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c0d2e50 0x7fbf2c0b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.019+0000 7fbf32f26640 1 -- 172.21.15.31:0/1767232788 >> 172.21.15.31:0/1767232788 conn(0x7fbf2c072310 msgr2=0x7fbf2c0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.019+0000 7fbf32f26640 1 -- 172.21.15.31:0/1767232788 shutdown_connections 2024-01-19T04:56:20.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.019+0000 7fbf32f26640 1 -- 172.21.15.31:0/1767232788 wait complete. 2024-01-19T04:56:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.020+0000 7fbf32f26640 1 Processor -- start 2024-01-19T04:56:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.020+0000 7fbf32f26640 1 -- start start 2024-01-19T04:56:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf2c0b4d00 0x7fbf2c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0d2e50 0x7fbf2c1f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c160a10 0x7fbf2c0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c15af20 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c15ad20 con 0x7fbf2c0b4d00 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c148f20 con 0x7fbf2c160a10 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0d2e50 0x7fbf2c1f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0d2e50 0x7fbf2c1f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57930/0 (socket says 172.21.15.31:57930) 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 -- 172.21.15.31:0/1367377831 learned_addr learned my addr 172.21.15.31:0/1367377831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf30cc2640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf2c0b4d00 0x7fbf2c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf314c3640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c160a10 0x7fbf2c0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 -- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c160a10 msgr2=0x7fbf2c0c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c160a10 0x7fbf2c0c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 -- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf2c0b4d00 msgr2=0x7fbf2c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf2c0b4d00 0x7fbf2c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 -- 172.21.15.31:0/1367377831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009b30 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf2bfff640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0d2e50 0x7fbf2c1f26d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c7f0 tx=0x7fbf1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf297fa640 1 -- 172.21.15.31:0/1367377831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800cea0 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf297fa640 1 -- 172.21.15.31:0/1367377831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf18007a30 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2c14c1f0 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf297fa640 1 -- 172.21.15.31:0/1367377831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18007dc0 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.021+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c159150 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.022+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2c0696e0 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.023+0000 7fbf297fa640 1 -- 172.21.15.31:0/1367377831 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fbf180113f0 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.023+0000 7fbf297fa640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf1006aed0 0x7fbf1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.023+0000 7fbf297fa640 1 -- 172.21.15.31:0/1367377831 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf18089390 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.024+0000 7fbf30cc2640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf1006aed0 0x7fbf1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.024+0000 7fbf30cc2640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf1006aed0 0x7fbf1006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf200016e0 tx=0x7fbf20009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.028+0000 7fbf297fa640 1 -- 172.21.15.31:0/1367377831 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbf18054660 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.276+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbf2c0d3ea0 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.277+0000 7fbf297fa640 1 -- 172.21.15.31:0/1367377831 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fbf1801f090 con 0x7fbf2c0d2e50 2024-01-19T04:56:20.276 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:20.276 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:56:20.276 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl9vwvARAAZduX/PeYlqDH8VDbvZiELg==", 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl6TSLDRAAGB2c1AX6e5HAuMDl6tCbKw==", 2024-01-19T04:56:20.277 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplMbeJCBAAkHOlhUk5I9r4TWFRo9L/eA==", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:56:20.278 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplQl/gFBAA3VbnYQWE/2FHLv7opt1f9w==", 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.279 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB7AKplbqKsNRAAikErjUtmLKvlkeH7GGL1UQ==", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB+AKplVZk5NBAA6FIuSbDo0CCaftcB9zQbmw==", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.280 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCBAKplP8CiMhAABJdd6kUn7DlZFCikXiggGQ==", 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCEAKplPAbxMBAANYNlBOGgzR60cDNfMpZTDA==", 2024-01-19T04:56:20.281 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCKAKplzlV2NxAAsprGSRQCPR98cdSNEBqmqA==", 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCNAKplqnZRNhAA2gX5iijb/OorAr9M0kxMBw==", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCQAKplG4yxNBAAIgP3l7C0BuCHY3ZzK0ezuQ==", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCTAKplKnckMhAApf2mP+F7NZ39VBHDkNVBTg==", 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw==", 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.284 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl11pPIhAAr+yfn0112ldu1NcYFzRIZg==", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl9grnHhAAnUsK7YtYaC8ZPOX4W84PbA==", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplog8YKBAA9SYGPnH5B4/9JInpxCxziw==", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-01-19T04:56:20.286 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3AKplRU9kIBAA1HYggMF7NTcKjXTzd1tRqg==", 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mds", 2024-01-19T04:56:20.287 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCeAKplICryKxAAT/+y3YoStcX50DQzpPmXqA==", 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mds" 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mgr", 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCeAKplCTTyKxAAeeQZlhqh5XXoZHwXEGY0SA==", 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-osd", 2024-01-19T04:56:20.288 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCeAKpl1DryKxAAz6zzr7QhS2UyMBeAWrix+w==", 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-osd" 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd", 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCeAKplYkHyKxAAQm5vz8tou9JkxtV6+IEUKQ==", 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-19T04:56:20.289 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCeAKplN0jyKxAAsX1j6y5tM+M9D8bz2xVnsw==", 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rgw", 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCeAKplhk/yKxAAAFqgZwbSuNeRi4X+IGp9PA==", 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3AKplwl9qORAAdjRFheF+sxCmZe31mpCjVw==", 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.290 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3AKpl6LqONRAAoCHvKhaqK1R2nkcFvmGb1A==", 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-01-19T04:56:20.291 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf1006aed0 msgr2=0x7fbf1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf1006aed0 0x7fbf1006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf200016e0 tx=0x7fbf20009040 comp rx=0 tx=0).stop 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0d2e50 msgr2=0x7fbf2c1f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0d2e50 0x7fbf2c1f26d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c7f0 tx=0x7fbf1800ccb0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 shutdown_connections 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf1006aed0 0x7fbf1006d390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf2c160a10 0x7fbf2c0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf2c0d2e50 0x7fbf2c1f26d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 --2- 172.21.15.31:0/1367377831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf2c0b4d00 0x7fbf2c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 >> 172.21.15.31:0/1367377831 conn(0x7fbf2c072310 msgr2=0x7fbf2c15e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 shutdown_connections 2024-01-19T04:56:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.279+0000 7fbf32f26640 1 -- 172.21.15.31:0/1367377831 wait complete. 2024-01-19T04:56:20.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-19T04:56:20.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.396+0000 7f55a5f1d640 1 Processor -- start 2024-01-19T04:56:20.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.396+0000 7f55a5f1d640 1 -- start start 2024-01-19T04:56:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f55a5f1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a013dfc0 0x7f55a015f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f55a5f1d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f55a015edd0 0x7f55a00cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f55a5f1d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55a00cf830 0x7f55a01408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f55a5f1d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a00cfce0 con 0x7f55a00cf830 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f55a5f1d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a006c640 con 0x7f55a015edd0 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f55a5f1d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a006c920 con 0x7f55a013dfc0 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55a00cf830 0x7f55a01408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55a00cf830 0x7f55a01408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57938/0 (socket says 172.21.15.31:57938) 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 -- 172.21.15.31:0/3928639813 learned_addr learned my addr 172.21.15.31:0/3928639813 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559effd640 1 --2- 172.21.15.31:0/3928639813 >> v2:172.21.15.137:3300/0 conn(0x7f55a015edd0 0x7f55a00cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559f7fe640 1 --2- 172.21.15.31:0/3928639813 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a013dfc0 0x7f55a015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 -- 172.21.15.31:0/3928639813 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a013dfc0 msgr2=0x7f55a015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 --2- 172.21.15.31:0/3928639813 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a013dfc0 0x7f55a015f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 -- 172.21.15.31:0/3928639813 >> v2:172.21.15.137:3300/0 conn(0x7f55a015edd0 msgr2=0x7f55a00cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 --2- 172.21.15.31:0/3928639813 >> v2:172.21.15.137:3300/0 conn(0x7f55a015edd0 0x7f55a00cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559ffff640 1 -- 172.21.15.31:0/3928639813 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a006cc00 con 0x7f55a00cf830 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559effd640 1 --2- 172.21.15.31:0/3928639813 >> v2:172.21.15.137:3300/0 conn(0x7f55a015edd0 0x7f55a00cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.397+0000 7f559f7fe640 1 --2- 172.21.15.31:0/3928639813 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a013dfc0 0x7f55a015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.398+0000 7f559ffff640 1 --2- 172.21.15.31:0/3928639813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a00cf830 0x7f55a01408e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55a00cff30 tx=0x7f559802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1214ce2ddf85777f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.398+0000 7f559e7fc640 1 -- 172.21.15.31:0/3928639813 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598031870 con 0x7f55a00cf830 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.398+0000 7f559e7fc640 1 -- 172.21.15.31:0/3928639813 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55980319d0 con 0x7f55a00cf830 2024-01-19T04:56:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.398+0000 7f559e7fc640 1 -- 172.21.15.31:0/3928639813 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598031d60 con 0x7f55a00cf830 2024-01-19T04:56:20.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 -- 172.21.15.31:0/3928639813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a00cf830 msgr2=0x7f55a01408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/3928639813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a00cf830 0x7f55a01408e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55a00cff30 tx=0x7f559802b910 comp rx=0 tx=0).stop 2024-01-19T04:56:20.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 -- 172.21.15.31:0/3928639813 shutdown_connections 2024-01-19T04:56:20.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/3928639813 >> v2:172.21.15.137:3300/0 conn(0x7f55a015edd0 0x7f55a00cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/3928639813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a00cf830 0x7f55a01408e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/3928639813 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a013dfc0 0x7f55a015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 -- 172.21.15.31:0/3928639813 >> 172.21.15.31:0/3928639813 conn(0x7f55a006aa00 msgr2=0x7f55a006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 -- 172.21.15.31:0/3928639813 shutdown_connections 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.399+0000 7f55a5f1d640 1 -- 172.21.15.31:0/3928639813 wait complete. 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.400+0000 7f55a5f1d640 1 Processor -- start 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.400+0000 7f55a5f1d640 1 -- start start 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f55a5f1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a00cf830 0x7f55a01f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f55a5f1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a013dfc0 0x7f55a01f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f55a5f1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a015edd0 0x7f55a00af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f55a5f1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a01553b0 con 0x7f55a013dfc0 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f55a5f1d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00b03b0 con 0x7f55a015edd0 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f55a5f1d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0154a10 con 0x7f55a00cf830 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a013dfc0 0x7f55a01f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a013dfc0 0x7f55a01f0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57948/0 (socket says 172.21.15.31:57948) 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 -- 172.21.15.31:0/2960180684 learned_addr learned my addr 172.21.15.31:0/2960180684 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:20.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559ffff640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a015edd0 0x7f55a00af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559f7fe640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a00cf830 0x7f55a01f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 -- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a00cf830 msgr2=0x7f55a01f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a00cf830 0x7f55a01f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 -- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a015edd0 msgr2=0x7f55a00af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a015edd0 0x7f55a00af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 -- 172.21.15.31:0/2960180684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55980094d0 con 0x7f55a013dfc0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559ffff640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a015edd0 0x7f55a00af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f559effd640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a013dfc0 0x7f55a01f0250 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f559400c7f0 tx=0x7f559400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.401+0000 7f5587fff640 1 -- 172.21.15.31:0/2960180684 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594007820 con 0x7f55a013dfc0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.402+0000 7f5587fff640 1 -- 172.21.15.31:0/2960180684 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5594007980 con 0x7f55a013dfc0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.402+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a0154cf0 con 0x7f55a013dfc0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.402+0000 7f5587fff640 1 -- 172.21.15.31:0/2960180684 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594007cf0 con 0x7f55a013dfc0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.402+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a0153f60 con 0x7f55a013dfc0 2024-01-19T04:56:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.403+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a0074900 con 0x7f55a013dfc0 2024-01-19T04:56:20.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.403+0000 7f5587fff640 1 -- 172.21.15.31:0/2960180684 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f55940113f0 con 0x7f55a013dfc0 2024-01-19T04:56:20.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.403+0000 7f5587fff640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f558c06aed0 0x7f558c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.404+0000 7f5587fff640 1 -- 172.21.15.31:0/2960180684 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55940894a0 con 0x7f55a013dfc0 2024-01-19T04:56:20.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.404+0000 7f559f7fe640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f558c06aed0 0x7f558c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.404+0000 7f559f7fe640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f558c06aed0 0x7f558c06d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55900075b0 tx=0x7f5590007540 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.408+0000 7f5587fff640 1 -- 172.21.15.31:0/2960180684 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5594054750 con 0x7f55a013dfc0 2024-01-19T04:56:20.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.653+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f55a015f7e0 con 0x7f55a013dfc0 2024-01-19T04:56:20.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.655+0000 7f5587fff640 1 -- 172.21.15.31:0/2960180684 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5594057d70 con 0x7f55a013dfc0 2024-01-19T04:56:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f558c06aed0 msgr2=0x7f558c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f558c06aed0 0x7f558c06d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55900075b0 tx=0x7f5590007540 comp rx=0 tx=0).stop 2024-01-19T04:56:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a013dfc0 msgr2=0x7f55a01f0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a013dfc0 0x7f55a01f0250 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f559400c7f0 tx=0x7f559400ccb0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 shutdown_connections 2024-01-19T04:56:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f558c06aed0 0x7f558c06d390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a015edd0 0x7f55a00af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a013dfc0 0x7f55a01f0250 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 --2- 172.21.15.31:0/2960180684 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55a00cf830 0x7f55a01f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.657+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 >> 172.21.15.31:0/2960180684 conn(0x7f55a006aa00 msgr2=0x7f55a0150590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.658+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 shutdown_connections 2024-01-19T04:56:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.658+0000 7f55a5f1d640 1 -- 172.21.15.31:0/2960180684 wait complete. 2024-01-19T04:56:20.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-19T04:56:20.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.770+0000 7f550a2a5640 1 Processor -- start 2024-01-19T04:56:20.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.770+0000 7f550a2a5640 1 -- start start 2024-01-19T04:56:20.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.770+0000 7f550a2a5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5504143440 0x7f55040cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.770+0000 7f550a2a5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550415ecb0 con 0x7f550406ddc0 2024-01-19T04:56:20.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.770+0000 7f550a2a5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550415e130 con 0x7f55040b33f0 2024-01-19T04:56:20.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.770+0000 7f550a2a5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5504072440 con 0x7f5504143440 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5508842640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f550406ddc0 0x7f55040b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39566/0 (socket says 172.21.15.31:39566) 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5508842640 1 -- 172.21.15.31:0/3164687558 learned_addr learned my addr 172.21.15.31:0/3164687558 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5503fff640 1 --2- 172.21.15.31:0/3164687558 >> v2:172.21.15.137:3300/0 conn(0x7f5504143440 0x7f55040cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874714985 0 0) 0x7f550415ecb0 con 0x7f550406ddc0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ec003520 con 0x7f550406ddc0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889115791 0 0) 0x7f550415e130 con 0x7f55040b33f0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550415ecb0 con 0x7f55040b33f0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2724602708 0 0) 0x7f54ec003520 con 0x7f550406ddc0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 >> v2:172.21.15.137:3300/0 conn(0x7f5504143440 msgr2=0x7f55040cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 --2- 172.21.15.31:0/3164687558 >> v2:172.21.15.137:3300/0 conn(0x7f5504143440 0x7f55040cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 >> v1:172.21.15.137:6789/0 conn(0x7f55040b33f0 legacy=0x7f550406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550415e410 con 0x7f550406ddc0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f54fc001970 con 0x7f550406ddc0 2024-01-19T04:56:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 >> v1:172.21.15.31:6789/0 conn(0x7f550406ddc0 legacy=0x7f55040b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54ec003b50 0x7f54ec003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec0047c0 0x7f54ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f54ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504072440 con 0x7f54ec0051d0 2024-01-19T04:56:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec00c100 con 0x7f54ec0047c0 2024-01-19T04:56:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec00c2c0 con 0x7f54ec003b50 2024-01-19T04:56:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5503fff640 1 --2- 172.21.15.31:0/3164687558 >> v2:172.21.15.137:3300/0 conn(0x7f5504143440 0x7f55040cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.771+0000 7f5508842640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f54ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f55037fe640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec0047c0 0x7f54ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5503fff640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54ec003b50 0x7f54ec003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5508842640 1 -- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54ec003b50 msgr2=0x7f54ec003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5508842640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54ec003b50 0x7f54ec003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5508842640 1 -- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec0047c0 msgr2=0x7f54ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5508842640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec0047c0 0x7f54ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5508842640 1 -- 172.21.15.31:0/3164687558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec003520 con 0x7f54ec0051d0 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5503fff640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f54ec003b50 0x7f54ec003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.772+0000 7f5508842640 1 -- 172.21.15.31:0/3164687558 reap_dead start 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f5508842640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f54ec00b980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00a690 tx=0x7f54ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682e96a6749072bd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc034630 con 0x7f54ec0051d0 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc02f070 con 0x7f54ec0051d0 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f5502ffd640 1 -- 172.21.15.31:0/3164687558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc034a10 con 0x7f54ec0051d0 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f550a2a5640 1 -- 172.21.15.31:0/3164687558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 msgr2=0x7f54ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f550a2a5640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f54ec00b980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00a690 tx=0x7f54ec0057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f550a2a5640 1 -- 172.21.15.31:0/3164687558 shutdown_connections 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f550a2a5640 1 --2- 172.21.15.31:0/3164687558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f54ec00b980 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f550a2a5640 1 -- 172.21.15.31:0/3164687558 >> 172.21.15.31:0/3164687558 conn(0x7f550406aa00 msgr2=0x7f550406ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f550a2a5640 1 -- 172.21.15.31:0/3164687558 shutdown_connections 2024-01-19T04:56:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.773+0000 7f550a2a5640 1 -- 172.21.15.31:0/3164687558 wait complete. 2024-01-19T04:56:20.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.774+0000 7f550a2a5640 1 Processor -- start 2024-01-19T04:56:20.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f550a2a5640 1 -- start start 2024-01-19T04:56:20.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f550a2a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f5504158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f550a2a5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f550413f800 0x7f55041410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f550a2a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55041587a0 0x7f550413e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f550a2a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504141630 con 0x7f54ec0051d0 2024-01-19T04:56:20.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f550a2a5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55041f2440 con 0x7f55041587a0 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f550a2a5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55041f2720 con 0x7f550413f800 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f5503fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f5504158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.775+0000 7f5503fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f5504158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57974/0 (socket says 172.21.15.31:57974) 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5503fff640 1 -- 172.21.15.31:0/1808050316 learned_addr learned my addr 172.21.15.31:0/1808050316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f55037fe640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f550413f800 0x7f55041410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5508842640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55041587a0 0x7f550413e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5503fff640 1 -- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f550413f800 msgr2=0x7f55041410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5503fff640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f550413f800 0x7f55041410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5503fff640 1 -- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55041587a0 msgr2=0x7f550413e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5503fff640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55041587a0 0x7f550413e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5503fff640 1 -- 172.21.15.31:0/1808050316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55041f2a00 con 0x7f54ec0051d0 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f55037fe640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f550413f800 0x7f55041410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5508842640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55041587a0 0x7f550413e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5503fff640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f5504158ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f54f4004650 tx=0x7f54f4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5500ff9640 1 -- 172.21.15.31:0/1808050316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400fbd0 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5500ff9640 1 -- 172.21.15.31:0/1808050316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f400fd30 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55041f2ce0 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f5500ff9640 1 -- 172.21.15.31:0/1808050316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4015660 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.776+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5504158c20 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.777+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550413fbe0 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.778+0000 7f5500ff9640 1 -- 172.21.15.31:0/1808050316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f54f40157c0 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.778+0000 7f5500ff9640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f54f006ae70 0x7f54f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.778+0000 7f5500ff9640 1 -- 172.21.15.31:0/1808050316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f54f4089cb0 con 0x7f54ec0051d0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.778+0000 7f55037fe640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f54f006ae70 0x7f54f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.779+0000 7f55037fe640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f54f006ae70 0x7f54f006d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f550415b0e0 tx=0x7f54f8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:20.782+0000 7f5500ff9640 1 -- 172.21.15.31:0/1808050316 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f54f4054ed0 con 0x7f54ec0051d0 2024-01-19T04:56:21.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.031+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f55041eedf0 con 0x7f54ec0051d0 2024-01-19T04:56:21.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.033+0000 7f5500ff9640 1 -- 172.21.15.31:0/1808050316 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f54f400fea0 con 0x7f54ec0051d0 2024-01-19T04:56:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f54f006ae70 msgr2=0x7f54f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f54f006ae70 0x7f54f006d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f550415b0e0 tx=0x7f54f8009040 comp rx=0 tx=0).stop 2024-01-19T04:56:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 msgr2=0x7f5504158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f5504158ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f54f4004650 tx=0x7f54f4011d60 comp rx=0 tx=0).stop 2024-01-19T04:56:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 shutdown_connections 2024-01-19T04:56:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f54f006ae70 0x7f54f006d330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55041587a0 0x7f550413e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f550413f800 0x7f55041410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 --2- 172.21.15.31:0/1808050316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54ec0051d0 0x7f5504158ef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 >> 172.21.15.31:0/1808050316 conn(0x7f550406aa00 msgr2=0x7f55040b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:21.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 shutdown_connections 2024-01-19T04:56:21.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.035+0000 7f550a2a5640 1 -- 172.21.15.31:0/1808050316 wait complete. 2024-01-19T04:56:21.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-19T04:56:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.148+0000 7f5f86b5a640 1 Processor -- start 2024-01-19T04:56:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.148+0000 7f5f86b5a640 1 -- start start 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f86b5a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8015b8c0 0x7f5f8015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f86b5a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f8013e8e0 0x7f5f8013ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f86b5a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5f80071a00 0x7f5f80071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f86b5a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8015fd50 con 0x7f5f8013e8e0 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f86b5a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f800723d0 con 0x7f5f80071a00 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f86b5a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8015f280 con 0x7f5f8015b8c0 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f8013e8e0 0x7f5f8013ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f8013e8e0 0x7f5f8013ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57984/0 (socket says 172.21.15.31:57984) 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 -- 172.21.15.31:0/1393774905 learned_addr learned my addr 172.21.15.31:0/1393774905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f850f7640 1 --2- 172.21.15.31:0/1393774905 >> v2:172.21.15.137:3300/0 conn(0x7f5f80071a00 0x7f5f80071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f848f6640 1 --2- 172.21.15.31:0/1393774905 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8015b8c0 0x7f5f8015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 -- 172.21.15.31:0/1393774905 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8015b8c0 msgr2=0x7f5f8015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 --2- 172.21.15.31:0/1393774905 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8015b8c0 0x7f5f8015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 -- 172.21.15.31:0/1393774905 >> v2:172.21.15.137:3300/0 conn(0x7f5f80071a00 msgr2=0x7f5f80071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 --2- 172.21.15.31:0/1393774905 >> v2:172.21.15.137:3300/0 conn(0x7f5f80071a00 0x7f5f80071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f7ffff640 1 -- 172.21.15.31:0/1393774905 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80071570 con 0x7f5f8013e8e0 2024-01-19T04:56:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.149+0000 7f5f850f7640 1 --2- 172.21.15.31:0/1393774905 >> v2:172.21.15.137:3300/0 conn(0x7f5f80071a00 0x7f5f80071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f7ffff640 1 --2- 172.21.15.31:0/1393774905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8013e8e0 0x7f5f8013ed60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5f8015ec70 tx=0x7f5f6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bb4901e88186bc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f7f7fe640 1 -- 172.21.15.31:0/1393774905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68031870 con 0x7f5f8013e8e0 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f7f7fe640 1 -- 172.21.15.31:0/1393774905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f680319d0 con 0x7f5f8013e8e0 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f7f7fe640 1 -- 172.21.15.31:0/1393774905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68031d60 con 0x7f5f8013e8e0 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1393774905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8013e8e0 msgr2=0x7f5f8013ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1393774905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8013e8e0 0x7f5f8013ed60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5f8015ec70 tx=0x7f5f6802b910 comp rx=0 tx=0).stop 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1393774905 shutdown_connections 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1393774905 >> v2:172.21.15.137:3300/0 conn(0x7f5f80071a00 0x7f5f80071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.150+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1393774905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8013e8e0 0x7f5f8013ed60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.151+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1393774905 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8015b8c0 0x7f5f8015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.151+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1393774905 >> 172.21.15.31:0/1393774905 conn(0x7f5f800b1c50 msgr2=0x7f5f800b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.151+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1393774905 shutdown_connections 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.151+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1393774905 wait complete. 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.152+0000 7f5f86b5a640 1 Processor -- start 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.152+0000 7f5f86b5a640 1 -- start start 2024-01-19T04:56:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.152+0000 7f5f86b5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f80071a00 0x7f5f80159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f86b5a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8013e8e0 0x7f5f800ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f86b5a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8015b8c0 0x7f5f8006e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f86b5a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800cfbc0 con 0x7f5f8015b8c0 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f86b5a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800cf040 con 0x7f5f80071a00 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f86b5a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800cf320 con 0x7f5f8013e8e0 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8015b8c0 0x7f5f8006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8015b8c0 0x7f5f8006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58000/0 (socket says 172.21.15.31:58000) 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 -- 172.21.15.31:0/1296701332 learned_addr learned my addr 172.21.15.31:0/1296701332 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f848f6640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f80071a00 0x7f5f80159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f7ffff640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8013e8e0 0x7f5f800ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 -- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8013e8e0 msgr2=0x7f5f800ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8013e8e0 0x7f5f800ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 -- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f80071a00 msgr2=0x7f5f80159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f80071a00 0x7f5f80159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 -- 172.21.15.31:0/1296701332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f680094d0 con 0x7f5f8015b8c0 2024-01-19T04:56:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f850f7640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8015b8c0 0x7f5f8006e700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5f78007e70 tx=0x7f5f7800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f7d7fa640 1 -- 172.21.15.31:0/1296701332 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78003b90 con 0x7f5f8015b8c0 2024-01-19T04:56:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f7d7fa640 1 -- 172.21.15.31:0/1296701332 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7800f040 con 0x7f5f8015b8c0 2024-01-19T04:56:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f7d7fa640 1 -- 172.21.15.31:0/1296701332 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800de80 con 0x7f5f8015b8c0 2024-01-19T04:56:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f800cf600 con 0x7f5f8015b8c0 2024-01-19T04:56:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.153+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f800c9dd0 con 0x7f5f8015b8c0 2024-01-19T04:56:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.154+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f80074900 con 0x7f5f8015b8c0 2024-01-19T04:56:21.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.155+0000 7f5f7d7fa640 1 -- 172.21.15.31:0/1296701332 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5f78004030 con 0x7f5f8015b8c0 2024-01-19T04:56:21.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.155+0000 7f5f7d7fa640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f6006aed0 0x7f5f6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.155+0000 7f5f7d7fa640 1 -- 172.21.15.31:0/1296701332 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5f78089560 con 0x7f5f8015b8c0 2024-01-19T04:56:21.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.155+0000 7f5f848f6640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f6006aed0 0x7f5f6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.156+0000 7f5f848f6640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f6006aed0 0x7f5f6006d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5f70006fd0 tx=0x7f5f70008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.160+0000 7f5f7d7fa640 1 -- 172.21.15.31:0/1296701332 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5f78054810 con 0x7f5f8015b8c0 2024-01-19T04:56:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.411+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f5f800727b0 con 0x7f5f8015b8c0 2024-01-19T04:56:21.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.413+0000 7f5f7d7fa640 1 -- 172.21.15.31:0/1296701332 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5f78057e30 con 0x7f5f8015b8c0 2024-01-19T04:56:21.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f6006aed0 msgr2=0x7f5f6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f6006aed0 0x7f5f6006d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5f70006fd0 tx=0x7f5f70008040 comp rx=0 tx=0).stop 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8015b8c0 msgr2=0x7f5f8006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8015b8c0 0x7f5f8006e700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5f78007e70 tx=0x7f5f7800d4a0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 shutdown_connections 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f6006aed0 0x7f5f6006d390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8015b8c0 0x7f5f8006e700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f8013e8e0 0x7f5f800ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 --2- 172.21.15.31:0/1296701332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f80071a00 0x7f5f80159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 >> 172.21.15.31:0/1296701332 conn(0x7f5f800b1c50 msgr2=0x7f5f8015af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 shutdown_connections 2024-01-19T04:56:21.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.416+0000 7f5f86b5a640 1 -- 172.21.15.31:0/1296701332 wait complete. 2024-01-19T04:56:21.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-19T04:56:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.529+0000 7f1d0ed0f640 1 Processor -- start 2024-01-19T04:56:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.529+0000 7f1d0ed0f640 1 -- start start 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0ed0f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d0815b420 0x7f1d0815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0ed0f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f1d0806fb00 0x7f1d0806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0ed0f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0806e870 con 0x7f1d080c3690 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0ed0f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0806eaa0 con 0x7f1d0806fb00 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0ed0f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806ed40 con 0x7f1d0815b420 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0caab640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d0815b420 0x7f1d0815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d07fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f1d0806fb00 0x7f1d0806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0caab640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d0815b420 0x7f1d0815edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:38798/0 (socket says 172.21.15.31:38798) 2024-01-19T04:56:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.530+0000 7f1d0caab640 1 -- 172.21.15.31:0/867367623 learned_addr learned my addr 172.21.15.31:0/867367623 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.531+0000 7f1d0caab640 1 -- 172.21.15.31:0/867367623 >> v2:172.21.15.137:3300/0 conn(0x7f1d0806fb00 msgr2=0x7f1d0806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.531+0000 7f1d0caab640 1 --2- 172.21.15.31:0/867367623 >> v2:172.21.15.137:3300/0 conn(0x7f1d0806fb00 0x7f1d0806a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.531+0000 7f1d0caab640 1 -- 172.21.15.31:0/867367623 >> v1:172.21.15.137:6789/0 conn(0x7f1d080c3690 legacy=0x7f1d080713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.531+0000 7f1d0caab640 1 -- 172.21.15.31:0/867367623 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0806f020 con 0x7f1d0815b420 2024-01-19T04:56:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.531+0000 7f1d077fe640 1 -- 172.21.15.31:0/867367623 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913725249 0 0) 0x7f1d0806e870 con 0x7f1d080c3690 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.531+0000 7f1d0caab640 1 --2- 172.21.15.31:0/867367623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d0815b420 0x7f1d0815edf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009870 tx=0x7f1cfc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96402b3244d1baf server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d077fe640 1 -- 172.21.15.31:0/867367623 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc031640 con 0x7f1d0815b420 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d077fe640 1 -- 172.21.15.31:0/867367623 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc0317a0 con 0x7f1d0815b420 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d077fe640 1 -- 172.21.15.31:0/867367623 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc031b30 con 0x7f1d0815b420 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/867367623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d0815b420 msgr2=0x7f1d0815edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/867367623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d0815b420 0x7f1d0815edf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009870 tx=0x7f1cfc02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/867367623 shutdown_connections 2024-01-19T04:56:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/867367623 >> v2:172.21.15.137:3300/0 conn(0x7f1d0806fb00 0x7f1d0806a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/867367623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d0815b420 0x7f1d0815edf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/867367623 >> 172.21.15.31:0/867367623 conn(0x7f1d080d0000 msgr2=0x7f1d080d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/867367623 shutdown_connections 2024-01-19T04:56:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.532+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/867367623 wait complete. 2024-01-19T04:56:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.533+0000 7f1d0ed0f640 1 Processor -- start 2024-01-19T04:56:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.533+0000 7f1d0ed0f640 1 -- start start 2024-01-19T04:56:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0ed0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d0806fb00 0x7f1d08159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0ed0f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d080c3690 0x7f1d080c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0ed0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d0815b420 0x7f1d080c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0ed0f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080ce2c0 con 0x7f1d0806fb00 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0ed0f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0815cbd0 con 0x7f1d0815b420 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0ed0f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080cdc70 con 0x7f1d080c3690 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0caab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d0806fb00 0x7f1d08159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0caab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d0806fb00 0x7f1d08159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58014/0 (socket says 172.21.15.31:58014) 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.534+0000 7f1d0caab640 1 -- 172.21.15.31:0/3612930563 learned_addr learned my addr 172.21.15.31:0/3612930563 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0d2ac640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d0815b420 0x7f1d080c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d07fff640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d080c3690 0x7f1d080c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0caab640 1 -- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d080c3690 msgr2=0x7f1d080c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0caab640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d080c3690 0x7f1d080c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0caab640 1 -- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d0815b420 msgr2=0x7f1d080c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0caab640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d0815b420 0x7f1d080c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0caab640 1 -- 172.21.15.31:0/3612930563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc0094d0 con 0x7f1d0806fb00 2024-01-19T04:56:21.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0caab640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d0806fb00 0x7f1d08159f10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009870 tx=0x7f1cfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d057fa640 1 -- 172.21.15.31:0/3612930563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc004230 con 0x7f1d0806fb00 2024-01-19T04:56:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d057fa640 1 -- 172.21.15.31:0/3612930563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc0043b0 con 0x7f1d0806fb00 2024-01-19T04:56:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d057fa640 1 -- 172.21.15.31:0/3612930563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc004520 con 0x7f1d0806fb00 2024-01-19T04:56:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0815e6f0 con 0x7f1d0806fb00 2024-01-19T04:56:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.535+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0815a450 con 0x7f1d0806fb00 2024-01-19T04:56:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.536+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d080ce400 con 0x7f1d0806fb00 2024-01-19T04:56:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.536+0000 7f1d057fa640 1 -- 172.21.15.31:0/3612930563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1cfc042070 con 0x7f1d0806fb00 2024-01-19T04:56:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.537+0000 7f1d057fa640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1cec06af00 0x7f1cec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.537+0000 7f1d057fa640 1 -- 172.21.15.31:0/3612930563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1cfc0aaf90 con 0x7f1d0806fb00 2024-01-19T04:56:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.537+0000 7f1d07fff640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1cec06af00 0x7f1cec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.538+0000 7f1d07fff640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1cec06af00 0x7f1cec06d3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d0806a8f0 tx=0x7f1cf4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.541+0000 7f1d057fa640 1 -- 172.21.15.31:0/3612930563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1cfc0de0d0 con 0x7f1d0806fb00 2024-01-19T04:56:21.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.789+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1d0806aab0 con 0x7f1d0806fb00 2024-01-19T04:56:21.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.791+0000 7f1d057fa640 1 -- 172.21.15.31:0/3612930563 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1d080ce400 con 0x7f1d0806fb00 2024-01-19T04:56:21.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.792+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1cec06af00 msgr2=0x7f1cec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.792+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1cec06af00 0x7f1cec06d3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d0806a8f0 tx=0x7f1cf4008040 comp rx=0 tx=0).stop 2024-01-19T04:56:21.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.792+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d0806fb00 msgr2=0x7f1d08159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.792+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d0806fb00 0x7f1d08159f10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009870 tx=0x7f1cfc0046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 shutdown_connections 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1cec06af00 0x7f1cec06d3c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d0815b420 0x7f1d080c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1d080c3690 0x7f1d080c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 --2- 172.21.15.31:0/3612930563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d0806fb00 0x7f1d08159f10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 >> 172.21.15.31:0/3612930563 conn(0x7f1d080d0000 msgr2=0x7f1d080a0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 shutdown_connections 2024-01-19T04:56:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.793+0000 7f1d0ed0f640 1 -- 172.21.15.31:0/3612930563 wait complete. 2024-01-19T04:56:21.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-19T04:56:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.905+0000 7f25e359e640 1 Processor -- start 2024-01-19T04:56:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.905+0000 7f25e359e640 1 -- start start 2024-01-19T04:56:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.906+0000 7f25e359e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f25e406b5c0 0x7f25e406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.906+0000 7f25e359e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e4156910 0x7f25e4156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.906+0000 7f25e359e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e406e220 con 0x7f25e40cef70 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.906+0000 7f25e359e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e406bf80 con 0x7f25e406b5c0 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.906+0000 7f25e359e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e406e6b0 con 0x7f25e4156910 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e2d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e4156910 0x7f25e4156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e1d9b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f25e406b5c0 0x7f25e406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e259c640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f25e40cef70 0x7f25e406cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:55604/0 (socket says 172.21.15.31:55604) 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e2d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e4156910 0x7f25e4156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:38822/0 (socket says 172.21.15.31:38822) 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e259c640 1 -- 172.21.15.31:0/2960048821 learned_addr learned my addr 172.21.15.31:0/2960048821 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e159a640 1 -- 172.21.15.31:0/2960048821 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337951439 0 0) 0x7f25e406e220 con 0x7f25e40cef70 2024-01-19T04:56:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e2d9d640 1 -- 172.21.15.31:0/2960048821 >> v2:172.21.15.137:3300/0 conn(0x7f25e406b5c0 msgr2=0x7f25e406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e2d9d640 1 --2- 172.21.15.31:0/2960048821 >> v2:172.21.15.137:3300/0 conn(0x7f25e406b5c0 0x7f25e406ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e2d9d640 1 -- 172.21.15.31:0/2960048821 >> v1:172.21.15.137:6789/0 conn(0x7f25e40cef70 legacy=0x7f25e406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.907+0000 7f25e2d9d640 1 -- 172.21.15.31:0/2960048821 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e415a010 con 0x7f25e4156910 2024-01-19T04:56:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.908+0000 7f25e2d9d640 1 --2- 172.21.15.31:0/2960048821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e4156910 0x7f25e4156da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25e406c0c0 tx=0x7f25dc02e0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8729b63dea52be1c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.908+0000 7f25e159a640 1 -- 172.21.15.31:0/2960048821 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc02ec40 con 0x7f25e4156910 2024-01-19T04:56:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.908+0000 7f25e159a640 1 -- 172.21.15.31:0/2960048821 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc02eda0 con 0x7f25e4156910 2024-01-19T04:56:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.908+0000 7f25e159a640 1 -- 172.21.15.31:0/2960048821 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc008660 con 0x7f25e4156910 2024-01-19T04:56:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.908+0000 7f25e359e640 1 -- 172.21.15.31:0/2960048821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e4156910 msgr2=0x7f25e4156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.908+0000 7f25e359e640 1 --2- 172.21.15.31:0/2960048821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e4156910 0x7f25e4156da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25e406c0c0 tx=0x7f25dc02e0a0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.909+0000 7f25e359e640 1 -- 172.21.15.31:0/2960048821 shutdown_connections 2024-01-19T04:56:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.909+0000 7f25e359e640 1 --2- 172.21.15.31:0/2960048821 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e4156910 0x7f25e4156da0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.909+0000 7f25e359e640 1 --2- 172.21.15.31:0/2960048821 >> v2:172.21.15.137:3300/0 conn(0x7f25e406b5c0 0x7f25e406ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.909+0000 7f25e359e640 1 -- 172.21.15.31:0/2960048821 >> 172.21.15.31:0/2960048821 conn(0x7f25e4070ab0 msgr2=0x7f25e4070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.909+0000 7f25e359e640 1 -- 172.21.15.31:0/2960048821 shutdown_connections 2024-01-19T04:56:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.909+0000 7f25e359e640 1 -- 172.21.15.31:0/2960048821 wait complete. 2024-01-19T04:56:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.910+0000 7f25e359e640 1 Processor -- start 2024-01-19T04:56:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.910+0000 7f25e359e640 1 -- start start 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25e406b5c0 0x7f25e4152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e40cef70 0x7f25e40b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e4156910 0x7f25e40b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e4154e00 con 0x7f25e4156910 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e4154b20 con 0x7f25e406b5c0 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40cc890 con 0x7f25e40cef70 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e4156910 0x7f25e40b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e4156910 0x7f25e40b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58024/0 (socket says 172.21.15.31:58024) 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 -- 172.21.15.31:0/2276097868 learned_addr learned my addr 172.21.15.31:0/2276097868 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e259c640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25e406b5c0 0x7f25e4152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e1d9b640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e40cef70 0x7f25e40b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 -- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e40cef70 msgr2=0x7f25e40b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e40cef70 0x7f25e40b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 -- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25e406b5c0 msgr2=0x7f25e4152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25e406b5c0 0x7f25e4152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 -- 172.21.15.31:0/2276097868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc00ab80 con 0x7f25e4156910 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e2d9d640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e4156910 0x7f25e40b3310 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f25dc038040 tx=0x7f25dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25daffd640 1 -- 172.21.15.31:0/2276097868 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc003b90 con 0x7f25e4156910 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25daffd640 1 -- 172.21.15.31:0/2276097868 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc003cf0 con 0x7f25e4156910 2024-01-19T04:56:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e40ccb70 con 0x7f25e4156910 2024-01-19T04:56:21.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25daffd640 1 -- 172.21.15.31:0/2276097868 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc004250 con 0x7f25e4156910 2024-01-19T04:56:21.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.911+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e4153180 con 0x7f25e4156910 2024-01-19T04:56:21.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.913+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e40cd030 con 0x7f25e4156910 2024-01-19T04:56:21.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.913+0000 7f25daffd640 1 -- 172.21.15.31:0/2276097868 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f25dc0043b0 con 0x7f25e4156910 2024-01-19T04:56:21.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.913+0000 7f25daffd640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f25c406aed0 0x7f25c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:21.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.913+0000 7f25daffd640 1 -- 172.21.15.31:0/2276097868 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25dc0adcb0 con 0x7f25e4156910 2024-01-19T04:56:21.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.913+0000 7f25e259c640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f25c406aed0 0x7f25c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:21.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.914+0000 7f25e259c640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f25c406aed0 0x7f25c406d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000f60 tx=0x7f25d4007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:21.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:21.917+0000 7f25daffd640 1 -- 172.21.15.31:0/2276097868 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f25dc078f80 con 0x7f25e4156910 2024-01-19T04:56:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.162+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f25e4074900 con 0x7f25e4156910 2024-01-19T04:56:22.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.164+0000 7f25daffd640 1 -- 172.21.15.31:0/2276097868 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f25dc00a560 con 0x7f25e4156910 2024-01-19T04:56:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f25c406aed0 msgr2=0x7f25c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f25c406aed0 0x7f25c406d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000f60 tx=0x7f25d4007d50 comp rx=0 tx=0).stop 2024-01-19T04:56:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e4156910 msgr2=0x7f25e40b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e4156910 0x7f25e40b3310 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f25dc038040 tx=0x7f25dc004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 shutdown_connections 2024-01-19T04:56:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f25c406aed0 0x7f25c406d390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e4156910 0x7f25e40b3310 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25e40cef70 0x7f25e40b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 --2- 172.21.15.31:0/2276097868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25e406b5c0 0x7f25e4152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 >> 172.21.15.31:0/2276097868 conn(0x7f25e4070ab0 msgr2=0x7f25e415a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 shutdown_connections 2024-01-19T04:56:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.166+0000 7f25e359e640 1 -- 172.21.15.31:0/2276097868 wait complete. 2024-01-19T04:56:22.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-19T04:56:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.279+0000 7fe74cde5640 1 Processor -- start 2024-01-19T04:56:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.279+0000 7fe74cde5640 1 -- start start 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe74cde5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe74806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe74cde5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7480cf570 0x7fe748161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe74cde5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7480cf9f0 con 0x7fe74815d780 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe74cde5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7480cfcd0 con 0x7fe7480cf570 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe74cde5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480cffb0 con 0x7fe74815d3a0 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe745d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7480cf570 0x7fe748161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe745d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7480cf570 0x7fe748161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58030/0 (socket says 172.21.15.31:58030) 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe745d9b640 1 -- 172.21.15.31:0/2267693022 learned_addr learned my addr 172.21.15.31:0/2267693022 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe74659c640 1 --2- 172.21.15.31:0/2267693022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe74806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe745d9b640 1 -- 172.21.15.31:0/2267693022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 msgr2=0x7fe74806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:22.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe745d9b640 1 --2- 172.21.15.31:0/2267693022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe74806c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe745d9b640 1 -- 172.21.15.31:0/2267693022 >> v1:172.21.15.137:6789/0 conn(0x7fe74815d780 legacy=0x7fe7480cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.280+0000 7fe745d9b640 1 -- 172.21.15.31:0/2267693022 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480d0290 con 0x7fe7480cf570 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.281+0000 7fe745d9b640 1 --2- 172.21.15.31:0/2267693022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480cf570 0x7fe748161150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009870 tx=0x7fe73c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a5564fd4f74de2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.281+0000 7fe74559a640 1 -- 172.21.15.31:0/2267693022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c031640 con 0x7fe7480cf570 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.281+0000 7fe74559a640 1 -- 172.21.15.31:0/2267693022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c0317a0 con 0x7fe7480cf570 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.281+0000 7fe74559a640 1 -- 172.21.15.31:0/2267693022 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c031b30 con 0x7fe7480cf570 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 -- 172.21.15.31:0/2267693022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480cf570 msgr2=0x7fe748161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 --2- 172.21.15.31:0/2267693022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480cf570 0x7fe748161150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009870 tx=0x7fe73c02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 -- 172.21.15.31:0/2267693022 shutdown_connections 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 --2- 172.21.15.31:0/2267693022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7480cf570 0x7fe748161150 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 --2- 172.21.15.31:0/2267693022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe74806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 -- 172.21.15.31:0/2267693022 >> 172.21.15.31:0/2267693022 conn(0x7fe7480d1100 msgr2=0x7fe7480d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 -- 172.21.15.31:0/2267693022 shutdown_connections 2024-01-19T04:56:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.282+0000 7fe74cde5640 1 -- 172.21.15.31:0/2267693022 wait complete. 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.283+0000 7fe74cde5640 1 Processor -- start 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.283+0000 7fe74cde5640 1 -- start start 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe74cde5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7480cf570 0x7fe7480b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe74cde5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe748152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe74cde5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815d780 0x7fe7481570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe74cde5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480c75d0 con 0x7fe74815d780 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe74cde5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7481526a0 con 0x7fe7480cf570 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe74cde5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe748156c60 con 0x7fe74815d3a0 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe746d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815d780 0x7fe7481570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe746d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815d780 0x7fe7481570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58044/0 (socket says 172.21.15.31:58044) 2024-01-19T04:56:22.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe746d9d640 1 -- 172.21.15.31:0/3399350514 learned_addr learned my addr 172.21.15.31:0/3399350514 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe74659c640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7480cf570 0x7fe7480b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.284+0000 7fe745d9b640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe748152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe746d9d640 1 -- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 msgr2=0x7fe748152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe746d9d640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe748152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe746d9d640 1 -- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7480cf570 msgr2=0x7fe7480b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe746d9d640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7480cf570 0x7fe7480b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe746d9d640 1 -- 172.21.15.31:0/3399350514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0094d0 con 0x7fe74815d780 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe74659c640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7480cf570 0x7fe7480b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe746d9d640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815d780 0x7fe7481570d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b970 tx=0x7fe73800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe736ffd640 1 -- 172.21.15.31:0/3399350514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73800cb40 con 0x7fe74815d780 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe736ffd640 1 -- 172.21.15.31:0/3399350514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73800cca0 con 0x7fe74815d780 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe748159370 con 0x7fe74815d780 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe736ffd640 1 -- 172.21.15.31:0/3399350514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738014680 con 0x7fe74815d780 2024-01-19T04:56:22.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.285+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe748159930 con 0x7fe74815d780 2024-01-19T04:56:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.286+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe748074900 con 0x7fe74815d780 2024-01-19T04:56:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.286+0000 7fe736ffd640 1 -- 172.21.15.31:0/3399350514 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe73801d3b0 con 0x7fe74815d780 2024-01-19T04:56:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.287+0000 7fe736ffd640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe72806aed0 0x7fe72806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.287+0000 7fe736ffd640 1 -- 172.21.15.31:0/3399350514 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7380891c0 con 0x7fe74815d780 2024-01-19T04:56:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.287+0000 7fe74659c640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe72806aed0 0x7fe72806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.288+0000 7fe74659c640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe72806aed0 0x7fe72806d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe74000a9b0 tx=0x7fe740008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.291+0000 7fe736ffd640 1 -- 172.21.15.31:0/3399350514 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe738054350 con 0x7fe74815d780 2024-01-19T04:56:22.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.538+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe748067030 con 0x7fe74815d780 2024-01-19T04:56:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.540+0000 7fe736ffd640 1 -- 172.21.15.31:0/3399350514 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe738087e60 con 0x7fe74815d780 2024-01-19T04:56:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.542+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe72806aed0 msgr2=0x7fe72806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.542+0000 7fe74cde5640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe72806aed0 0x7fe72806d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe74000a9b0 tx=0x7fe740008040 comp rx=0 tx=0).stop 2024-01-19T04:56:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.542+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815d780 msgr2=0x7fe7481570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.542+0000 7fe74cde5640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815d780 0x7fe7481570d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b970 tx=0x7fe73800be30 comp rx=0 tx=0).stop 2024-01-19T04:56:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 shutdown_connections 2024-01-19T04:56:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe72806aed0 0x7fe72806d390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815d780 0x7fe7481570d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe74815d3a0 0x7fe748152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 --2- 172.21.15.31:0/3399350514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7480cf570 0x7fe7480b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 >> 172.21.15.31:0/3399350514 conn(0x7fe7480d1100 msgr2=0x7fe7481603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 shutdown_connections 2024-01-19T04:56:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.543+0000 7fe74cde5640 1 -- 172.21.15.31:0/3399350514 wait complete. 2024-01-19T04:56:22.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:22.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-19T04:56:22.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.654+0000 7f8e2a61d640 1 Processor -- start 2024-01-19T04:56:22.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.654+0000 7f8e2a61d640 1 -- start start 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e2a61d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e240cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e2a61d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e240c9520 0x7f8e240cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e2a61d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e240cbce0 con 0x7f8e2415d610 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e2a61d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e240cd300 con 0x7f8e240c9520 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e2a61d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e24161400 con 0x7f8e2415d230 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e28bba640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e240c9520 0x7f8e240cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e237fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8e2415d610 0x7f8e2406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39574/0 (socket says 172.21.15.31:39574) 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e237fe640 1 -- 172.21.15.31:0/1890898346 learned_addr learned my addr 172.21.15.31:0/1890898346 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e28bba640 1 -- 172.21.15.31:0/1890898346 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 msgr2=0x7f8e240cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e28bba640 1 --2- 172.21.15.31:0/1890898346 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e240cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e28bba640 1 -- 172.21.15.31:0/1890898346 >> v1:172.21.15.31:6789/0 conn(0x7f8e2415d610 legacy=0x7f8e2406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.656+0000 7f8e28bba640 1 -- 172.21.15.31:0/1890898346 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e241616e0 con 0x7f8e240c9520 2024-01-19T04:56:22.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.655+0000 7f8e22ffd640 1 -- 172.21.15.31:0/1890898346 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666982260 0 0) 0x7f8e240cbce0 con 0x7f8e2415d610 2024-01-19T04:56:22.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.656+0000 7f8e28bba640 1 --2- 172.21.15.31:0/1890898346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e240c9520 0x7f8e240cb7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009d90 tx=0x7f8e1c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e66ca88a37f50c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.656+0000 7f8e22ffd640 1 -- 172.21.15.31:0/1890898346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c02cc50 con 0x7f8e240c9520 2024-01-19T04:56:22.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.656+0000 7f8e22ffd640 1 -- 172.21.15.31:0/1890898346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c02cdb0 con 0x7f8e240c9520 2024-01-19T04:56:22.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.656+0000 7f8e22ffd640 1 -- 172.21.15.31:0/1890898346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c034660 con 0x7f8e240c9520 2024-01-19T04:56:22.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 -- 172.21.15.31:0/1890898346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e240c9520 msgr2=0x7f8e240cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/1890898346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e240c9520 0x7f8e240cb7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009d90 tx=0x7f8e1c02bea0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 -- 172.21.15.31:0/1890898346 shutdown_connections 2024-01-19T04:56:22.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/1890898346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e240c9520 0x7f8e240cb7a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/1890898346 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e240cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 -- 172.21.15.31:0/1890898346 >> 172.21.15.31:0/1890898346 conn(0x7f8e240cc1b0 msgr2=0x7f8e240cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:22.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 -- 172.21.15.31:0/1890898346 shutdown_connections 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.657+0000 7f8e2a61d640 1 -- 172.21.15.31:0/1890898346 wait complete. 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.658+0000 7f8e2a61d640 1 Processor -- start 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.658+0000 7f8e2a61d640 1 -- start start 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e2a61d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e240c9520 0x7f8e241602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e2a61d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e241508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e2a61d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e2415d610 0x7f8e241f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e2a61d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240b4460 con 0x7f8e2415d610 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e2a61d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e24160830 con 0x7f8e240c9520 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e2a61d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e24150e10 con 0x7f8e2415d230 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e2415d610 0x7f8e241f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e2415d610 0x7f8e241f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58058/0 (socket says 172.21.15.31:58058) 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 -- 172.21.15.31:0/3078962171 learned_addr learned my addr 172.21.15.31:0/3078962171 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e237fe640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e241508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e23fff640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e240c9520 0x7f8e241602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 -- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 msgr2=0x7f8e241508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e241508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 -- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e240c9520 msgr2=0x7f8e241602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e240c9520 0x7f8e241602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e28bba640 1 -- 172.21.15.31:0/3078962171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009a70 con 0x7f8e2415d610 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e237fe640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e241508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.659+0000 7f8e23fff640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e240c9520 0x7f8e241602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.660+0000 7f8e28bba640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e2415d610 0x7f8e241f3120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009d90 tx=0x7f8e1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.660+0000 7f8e20ff9640 1 -- 172.21.15.31:0/3078962171 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c03e610 con 0x7f8e2415d610 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.660+0000 7f8e20ff9640 1 -- 172.21.15.31:0/3078962171 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c03e790 con 0x7f8e2415d610 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.660+0000 7f8e20ff9640 1 -- 172.21.15.31:0/3078962171 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c03eb40 con 0x7f8e2415d610 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.660+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e240b3850 con 0x7f8e2415d610 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.660+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e240b3e10 con 0x7f8e2415d610 2024-01-19T04:56:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.661+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e240b45d0 con 0x7f8e2415d610 2024-01-19T04:56:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.661+0000 7f8e20ff9640 1 -- 172.21.15.31:0/3078962171 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8e1c03b050 con 0x7f8e2415d610 2024-01-19T04:56:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.661+0000 7f8e20ff9640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006aed0 0x7f8e1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.662+0000 7f8e20ff9640 1 -- 172.21.15.31:0/3078962171 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e1c0abcb0 con 0x7f8e2415d610 2024-01-19T04:56:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.662+0000 7f8e23fff640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006aed0 0x7f8e1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.662+0000 7f8e23fff640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006aed0 0x7f8e1006d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009930 tx=0x7f8e18008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.666+0000 7f8e20ff9640 1 -- 172.21.15.31:0/3078962171 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8e1c076ed0 con 0x7f8e2415d610 2024-01-19T04:56:22.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.846+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e240696e0 con 0x7f8e2415d610 2024-01-19T04:56:22.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.847+0000 7f8e20ff9640 1 -- 172.21.15.31:0/3078962171 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12822 (secure 0 0 0) 0x7f8e1c04e090 con 0x7f8e2415d610 2024-01-19T04:56:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:22.846 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":30,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:19.891651+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006aed0 msgr2=0x7f8e1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006aed0 0x7f8e1006d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009930 tx=0x7f8e18008040 comp rx=0 tx=0).stop 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e2415d610 msgr2=0x7f8e241f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e2415d610 0x7f8e241f3120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009d90 tx=0x7f8e1c004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 shutdown_connections 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006aed0 0x7f8e1006d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e2415d610 0x7f8e241f3120 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e2415d230 0x7f8e241508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 --2- 172.21.15.31:0/3078962171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e240c9520 0x7f8e241602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 >> 172.21.15.31:0/3078962171 conn(0x7f8e240cc1b0 msgr2=0x7f8e2406eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:22.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 shutdown_connections 2024-01-19T04:56:22.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.849+0000 7f8e2a61d640 1 -- 172.21.15.31:0/3078962171 wait complete. 2024-01-19T04:56:22.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:22.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.965+0000 7fb3d1e2f640 1 Processor -- start 2024-01-19T04:56:22.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.965+0000 7fb3d1e2f640 1 -- start start 2024-01-19T04:56:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.965+0000 7fb3d1e2f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3cc13dfc0 0x7fb3cc15f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.965+0000 7fb3d1e2f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc0cfa00 0x7fb3cc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.965+0000 7fb3d1e2f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc06c640 con 0x7fb3cc15edd0 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.965+0000 7fb3d1e2f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc06c920 con 0x7fb3cc13dfc0 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.965+0000 7fb3d1e2f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc06cc00 con 0x7fb3cc0cfa00 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3cb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3cc13dfc0 0x7fb3cc15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3caffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb3cc15edd0 0x7fb3cc0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39578/0 (socket says 172.21.15.31:39578) 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3caffd640 1 -- 172.21.15.31:0/1261952642 learned_addr learned my addr 172.21.15.31:0/1261952642 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3cb7fe640 1 -- 172.21.15.31:0/1261952642 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc0cfa00 msgr2=0x7fb3cc0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3cbfff640 1 --2- 172.21.15.31:0/1261952642 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc0cfa00 0x7fb3cc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3ca7fc640 1 -- 172.21.15.31:0/1261952642 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637694969 0 0) 0x7fb3cc06c640 con 0x7fb3cc15edd0 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3cb7fe640 1 --2- 172.21.15.31:0/1261952642 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc0cfa00 0x7fb3cc0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3cb7fe640 1 -- 172.21.15.31:0/1261952642 >> v1:172.21.15.31:6789/0 conn(0x7fb3cc15edd0 legacy=0x7fb3cc0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3cb7fe640 1 -- 172.21.15.31:0/1261952642 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc06cee0 con 0x7fb3cc13dfc0 2024-01-19T04:56:22.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.966+0000 7fb3cbfff640 1 --2- 172.21.15.31:0/1261952642 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc0cfa00 0x7fb3cc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:22.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3cb7fe640 1 --2- 172.21.15.31:0/1261952642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc13dfc0 0x7fb3cc15f520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00097a0 tx=0x7fb3c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769f8563eb607cd4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3ca7fc640 1 -- 172.21.15.31:0/1261952642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0031830 con 0x7fb3cc13dfc0 2024-01-19T04:56:22.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3ca7fc640 1 -- 172.21.15.31:0/1261952642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0031990 con 0x7fb3cc13dfc0 2024-01-19T04:56:22.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3ca7fc640 1 -- 172.21.15.31:0/1261952642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0031d20 con 0x7fb3cc13dfc0 2024-01-19T04:56:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/1261952642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc13dfc0 msgr2=0x7fb3cc15f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/1261952642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc13dfc0 0x7fb3cc15f520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00097a0 tx=0x7fb3c002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/1261952642 shutdown_connections 2024-01-19T04:56:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/1261952642 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc0cfa00 0x7fb3cc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/1261952642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc13dfc0 0x7fb3cc15f520 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/1261952642 >> 172.21.15.31:0/1261952642 conn(0x7fb3cc06aa00 msgr2=0x7fb3cc06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/1261952642 shutdown_connections 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.967+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/1261952642 wait complete. 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 Processor -- start 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 -- start start 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3cc0cfa00 0x7fb3cc1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc13dfc0 0x7fb3cc150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc15edd0 0x7fb3cc150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0aa720 con 0x7fb3cc15edd0 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc150600 con 0x7fb3cc0cfa00 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.969+0000 7fb3d1e2f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0a9370 con 0x7fb3cc13dfc0 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc15edd0 0x7fb3cc150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc15edd0 0x7fb3cc150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58066/0 (socket says 172.21.15.31:58066) 2024-01-19T04:56:22.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 -- 172.21.15.31:0/4113053680 learned_addr learned my addr 172.21.15.31:0/4113053680 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cb7fe640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3cc0cfa00 0x7fb3cc1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3caffd640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc13dfc0 0x7fb3cc150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 -- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc13dfc0 msgr2=0x7fb3cc150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc13dfc0 0x7fb3cc150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 -- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3cc0cfa00 msgr2=0x7fb3cc1500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3cc0cfa00 0x7fb3cc1500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 -- 172.21.15.31:0/4113053680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0009400 con 0x7fb3cc15edd0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3cbfff640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc15edd0 0x7fb3cc150fb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400ce80 tx=0x7fb3c400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3b3fff640 1 -- 172.21.15.31:0/4113053680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400d840 con 0x7fb3cc15edd0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3b3fff640 1 -- 172.21.15.31:0/4113053680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c400d9e0 con 0x7fb3cc15edd0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc0aa060 con 0x7fb3cc15edd0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3b3fff640 1 -- 172.21.15.31:0/4113053680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400dd90 con 0x7fb3cc15edd0 2024-01-19T04:56:22.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.970+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc0a9650 con 0x7fb3cc15edd0 2024-01-19T04:56:22.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.971+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc0aa860 con 0x7fb3cc15edd0 2024-01-19T04:56:22.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.972+0000 7fb3b3fff640 1 -- 172.21.15.31:0/4113053680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb3c4015440 con 0x7fb3cc15edd0 2024-01-19T04:56:22.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.972+0000 7fb3b3fff640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3b806aed0 0x7fb3b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.972+0000 7fb3b3fff640 1 -- 172.21.15.31:0/4113053680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb3c4089b40 con 0x7fb3cc15edd0 2024-01-19T04:56:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.973+0000 7fb3cb7fe640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3b806aed0 0x7fb3b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:22.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.973+0000 7fb3cb7fe640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3b806aed0 0x7fb3b806d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0036000 tx=0x7fb3c003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:22.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:22.977+0000 7fb3b3fff640 1 -- 172.21.15.31:0/4113053680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb3cc0aa860 con 0x7fb3cc15edd0 2024-01-19T04:56:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.153+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3cc0aa860 con 0x7fb3cc15edd0 2024-01-19T04:56:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.154+0000 7fb3b3fff640 1 -- 172.21.15.31:0/4113053680 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3cc0aa860 con 0x7fb3cc15edd0 2024-01-19T04:56:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.155+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3b806aed0 msgr2=0x7fb3b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.155+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3b806aed0 0x7fb3b806d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0036000 tx=0x7fb3c003c040 comp rx=0 tx=0).stop 2024-01-19T04:56:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc15edd0 msgr2=0x7fb3cc150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc15edd0 0x7fb3cc150fb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400ce80 tx=0x7fb3c400a710 comp rx=0 tx=0).stop 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 shutdown_connections 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3b806aed0 0x7fb3b806d390 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc15edd0 0x7fb3cc150fb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3cc13dfc0 0x7fb3cc150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 --2- 172.21.15.31:0/4113053680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3cc0cfa00 0x7fb3cc1500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 >> 172.21.15.31:0/4113053680 conn(0x7fb3cc06aa00 msgr2=0x7fb3cc06ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 shutdown_connections 2024-01-19T04:56:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.156+0000 7fb3d1e2f640 1 -- 172.21.15.31:0/4113053680 wait complete. 2024-01-19T04:56:23.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-19T04:56:23.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.267+0000 7f2ba1b0e640 1 Processor -- start 2024-01-19T04:56:23.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.267+0000 7f2ba1b0e640 1 -- start start 2024-01-19T04:56:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2ba1b0e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2b9c160bb0 0x7f2b9c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2ba1b0e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0cfd10 0x7f2b9c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2ba1b0e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0d06b0 con 0x7f2b9c0d0ef0 2024-01-19T04:56:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2ba1b0e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b66b0 con 0x7f2b9c160bb0 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2ba1b0e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0d1b90 con 0x7f2b9c0cfd10 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2b9affd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0cfd10 0x7f2b9c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2b9b7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2b9c160bb0 0x7f2b9c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2b9bfff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f2b9c0d0ef0 0x7f2b9c0d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:55614/0 (socket says 172.21.15.31:55614) 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2b9bfff640 1 -- 172.21.15.31:0/949115293 learned_addr learned my addr 172.21.15.31:0/949115293 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.268+0000 7f2b9affd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0cfd10 0x7f2b9c0d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:38886/0 (socket says 172.21.15.31:38886) 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.269+0000 7f2b9affd640 1 -- 172.21.15.31:0/949115293 >> v2:172.21.15.137:3300/0 conn(0x7f2b9c160bb0 msgr2=0x7f2b9c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.269+0000 7f2b9affd640 1 --2- 172.21.15.31:0/949115293 >> v2:172.21.15.137:3300/0 conn(0x7f2b9c160bb0 0x7f2b9c0b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.269+0000 7f2b9affd640 1 -- 172.21.15.31:0/949115293 >> v1:172.21.15.137:6789/0 conn(0x7f2b9c0d0ef0 legacy=0x7f2b9c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.269+0000 7f2b9affd640 1 -- 172.21.15.31:0/949115293 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0d1e70 con 0x7f2b9c0cfd10 2024-01-19T04:56:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.269+0000 7f2b9b7fe640 1 --2- 172.21.15.31:0/949115293 >> v2:172.21.15.137:3300/0 conn(0x7f2b9c160bb0 0x7f2b9c0b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.269+0000 7f2b9affd640 1 --2- 172.21.15.31:0/949115293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0cfd10 0x7f2b9c0d0170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b900097a0 tx=0x7f2b9002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccca325a2a39fe91 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2b9a7fc640 1 -- 172.21.15.31:0/949115293 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90031830 con 0x7f2b9c0cfd10 2024-01-19T04:56:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2b9a7fc640 1 -- 172.21.15.31:0/949115293 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90031990 con 0x7f2b9c0cfd10 2024-01-19T04:56:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2b9a7fc640 1 -- 172.21.15.31:0/949115293 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90031d20 con 0x7f2b9c0cfd10 2024-01-19T04:56:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/949115293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0cfd10 msgr2=0x7f2b9c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/949115293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0cfd10 0x7f2b9c0d0170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b900097a0 tx=0x7f2b9002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/949115293 shutdown_connections 2024-01-19T04:56:23.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/949115293 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0cfd10 0x7f2b9c0d0170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/949115293 >> v2:172.21.15.137:3300/0 conn(0x7f2b9c160bb0 0x7f2b9c0b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/949115293 >> 172.21.15.31:0/949115293 conn(0x7f2b9c09c600 msgr2=0x7f2b9c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:23.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/949115293 shutdown_connections 2024-01-19T04:56:23.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.270+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/949115293 wait complete. 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.271+0000 7f2ba1b0e640 1 Processor -- start 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2ba1b0e640 1 -- start start 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2ba1b0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b9c0cfd10 0x7f2b9c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2ba1b0e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0d0ef0 0x7f2b9c15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2ba1b0e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9c160bb0 0x7f2b9c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2ba1b0e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1f3240 con 0x7f2b9c160bb0 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2ba1b0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0c76f0 con 0x7f2b9c0cfd10 2024-01-19T04:56:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2ba1b0e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c160290 con 0x7f2b9c0d0ef0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9c160bb0 0x7f2b9c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9c160bb0 0x7f2b9c15f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58080/0 (socket says 172.21.15.31:58080) 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2b9bfff640 1 -- 172.21.15.31:0/3093768297 learned_addr learned my addr 172.21.15.31:0/3093768297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.272+0000 7f2b9affd640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0d0ef0 0x7f2b9c15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b9b7fe640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b9c0cfd10 0x7f2b9c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b9bfff640 1 -- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0d0ef0 msgr2=0x7f2b9c15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b9bfff640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0d0ef0 0x7f2b9c15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b9bfff640 1 -- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b9c0cfd10 msgr2=0x7f2b9c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b9bfff640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b9c0cfd10 0x7f2b9c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b9bfff640 1 -- 172.21.15.31:0/3093768297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c009400 con 0x7f2b9c160bb0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b9bfff640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9c160bb0 0x7f2b9c15f730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400ca80 tx=0x7f2b9400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b83fff640 1 -- 172.21.15.31:0/3093768297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400ac30 con 0x7f2b9c160bb0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b83fff640 1 -- 172.21.15.31:0/3093768297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9400ad90 con 0x7f2b9c160bb0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b90009400 con 0x7f2b9c160bb0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2b83fff640 1 -- 172.21.15.31:0/3093768297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94012600 con 0x7f2b9c160bb0 2024-01-19T04:56:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.273+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c0c7410 con 0x7f2b9c160bb0 2024-01-19T04:56:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.274+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c0caf40 con 0x7f2b9c160bb0 2024-01-19T04:56:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.275+0000 7f2b83fff640 1 -- 172.21.15.31:0/3093768297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2b94007070 con 0x7f2b9c160bb0 2024-01-19T04:56:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.275+0000 7f2b83fff640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2b8806aed0 0x7f2b8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.275+0000 7f2b83fff640 1 -- 172.21.15.31:0/3093768297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2b94089010 con 0x7f2b9c160bb0 2024-01-19T04:56:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.275+0000 7f2b9b7fe640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2b8806aed0 0x7f2b8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.276+0000 7f2b9b7fe640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2b8806aed0 0x7f2b8806d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009cd0 tx=0x7f2b8c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:23.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.279+0000 7f2b83fff640 1 -- 172.21.15.31:0/3093768297 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2b94054370 con 0x7f2b9c160bb0 2024-01-19T04:56:23.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.459+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2b9c0696e0 con 0x7f2b9c160bb0 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.459+0000 7f2b83fff640 1 -- 172.21.15.31:0/3093768297 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2b94017060 con 0x7f2b9c160bb0 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:23.459 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2b8806aed0 msgr2=0x7f2b8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2b8806aed0 0x7f2b8806d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009cd0 tx=0x7f2b8c009190 comp rx=0 tx=0).stop 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9c160bb0 msgr2=0x7f2b9c15f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9c160bb0 0x7f2b9c15f730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400ca80 tx=0x7f2b9400cf40 comp rx=0 tx=0).stop 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 shutdown_connections 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2b8806aed0 0x7f2b8806d390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9c160bb0 0x7f2b9c15f730 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b9c0d0ef0 0x7f2b9c15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 --2- 172.21.15.31:0/3093768297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b9c0cfd10 0x7f2b9c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 >> 172.21.15.31:0/3093768297 conn(0x7f2b9c09c600 msgr2=0x7f2b9c0d3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 shutdown_connections 2024-01-19T04:56:23.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.461+0000 7f2ba1b0e640 1 -- 172.21.15.31:0/3093768297 wait complete. 2024-01-19T04:56:23.471 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-19T04:56:23.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-19T04:56:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.576+0000 7f4e6f7b0640 1 Processor -- start 2024-01-19T04:56:23.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.577+0000 7f4e6f7b0640 1 -- start start 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.577+0000 7f4e6f7b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e6815b870 0x7f4e680b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.577+0000 7f4e6f7b0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e680703c0 con 0x7f4e68144a20 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.577+0000 7f4e6f7b0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e680b17f0 con 0x7f4e680b13c0 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.577+0000 7f4e6f7b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e68144e00 con 0x7f4e6815b870 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e6d54c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4e68144a20 0x7f4e680b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39592/0 (socket says 172.21.15.31:39592) 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e6d54c640 1 -- 172.21.15.31:0/3613146690 learned_addr learned my addr 172.21.15.31:0/3613146690 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256565354 0 0) 0x7f4e680703c0 con 0x7f4e68144a20 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e54003520 con 0x7f4e68144a20 2024-01-19T04:56:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e6815b870 0x7f4e680b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2024972226 0 0) 0x7f4e54003520 con 0x7f4e68144a20 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e6815b870 msgr2=0x7f4e680b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e6815b870 0x7f4e680b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 >> v1:172.21.15.137:6789/0 conn(0x7f4e680b13c0 legacy=0x7f4e680cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680b48b0 con 0x7f4e68144a20 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f4e64001670 con 0x7f4e68144a20 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 >> v1:172.21.15.31:6789/0 conn(0x7f4e68144a20 legacy=0x7f4e680b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e540039a0 0x7f4e54003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e540047c0 0x7f4e54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540052a0 0x7f4e5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e68144e00 con 0x7f4e540047c0 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5400c0d0 con 0x7f4e540039a0 2024-01-19T04:56:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5400c290 con 0x7f4e540052a0 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e6815b870 0x7f4e680b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e540047c0 0x7f4e54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e6d54c640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e540039a0 0x7f4e54003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.578+0000 7f4e6dd4d640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540052a0 0x7f4e5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e6cd4b640 1 -- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540052a0 msgr2=0x7f4e5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540052a0 0x7f4e5400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e6cd4b640 1 -- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e540039a0 msgr2=0x7f4e54003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e540039a0 0x7f4e54003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e6cd4b640 1 -- 172.21.15.31:0/3613146690 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e54003520 con 0x7f4e540047c0 2024-01-19T04:56:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e6dd4d640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540052a0 0x7f4e5400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e540047c0 0x7f4e54004c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006fd0 tx=0x7f4e540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96c3e3d1d2693bb server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58033600 con 0x7f4e540047c0 2024-01-19T04:56:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5802f050 con 0x7f4e540047c0 2024-01-19T04:56:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.579+0000 7f4e5ffff640 1 -- 172.21.15.31:0/3613146690 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58033980 con 0x7f4e540047c0 2024-01-19T04:56:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.580+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/3613146690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e540047c0 msgr2=0x7f4e54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.580+0000 7f4e6f7b0640 1 --2- 172.21.15.31:0/3613146690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e540047c0 0x7f4e54004c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006fd0 tx=0x7f4e540057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.580+0000 7f4e6dd4d640 1 -- 172.21.15.31:0/3613146690 reap_dead start 2024-01-19T04:56:23.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.580+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/3613146690 shutdown_connections 2024-01-19T04:56:23.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.580+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/3613146690 >> 172.21.15.31:0/3613146690 conn(0x7f4e6806a790 msgr2=0x7f4e6806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:23.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.580+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/3613146690 shutdown_connections 2024-01-19T04:56:23.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.580+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/3613146690 wait complete. 2024-01-19T04:56:23.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.581+0000 7f4e6f7b0640 1 Processor -- start 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.581+0000 7f4e6f7b0640 1 -- start start 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6f7b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540047c0 0x7f4e6813a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6f7b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6813b2c0 0x7f4e680af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6f7b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e680af600 0x7f4e680ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6f7b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680afa80 con 0x7f4e6813b2c0 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6f7b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680cd4f0 con 0x7f4e680af600 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6f7b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680ce990 con 0x7f4e540047c0 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6813b2c0 0x7f4e680af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6813b2c0 0x7f4e680af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58090/0 (socket says 172.21.15.31:58090) 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 -- 172.21.15.31:0/647972938 learned_addr learned my addr 172.21.15.31:0/647972938 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 -- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540047c0 msgr2=0x7f4e6813a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540047c0 0x7f4e6813a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6dd4d640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e680af600 0x7f4e680ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 -- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e680af600 msgr2=0x7f4e680ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e680af600 0x7f4e680ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 -- 172.21.15.31:0/647972938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680cd740 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.582+0000 7f4e6cd4b640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6813b2c0 0x7f4e680af0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e5802bdc0 tx=0x7f4e58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.583+0000 7f4e5dffb640 1 -- 172.21.15.31:0/647972938 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e580328f0 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.583+0000 7f4e5dffb640 1 -- 172.21.15.31:0/647972938 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e58032a50 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.583+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e680cda20 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.583+0000 7f4e5dffb640 1 -- 172.21.15.31:0/647972938 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58032de0 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.583+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6813b740 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.584+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e680af9e0 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.584+0000 7f4e5dffb640 1 -- 172.21.15.31:0/647972938 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4e5803d410 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.584+0000 7f4e5dffb640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e4c06aed0 0x7f4e4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.585+0000 7f4e5dffb640 1 -- 172.21.15.31:0/647972938 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e580ab5d0 con 0x7f4e6813b2c0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.585+0000 7f4e6d54c640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e4c06aed0 0x7f4e4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:23.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.585+0000 7f4e6d54c640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e4c06aed0 0x7f4e4c06d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e64001c90 tx=0x7f4e64001b40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.589+0000 7f4e5dffb640 1 -- 172.21.15.31:0/647972938 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4e58076930 con 0x7f4e6813b2c0 2024-01-19T04:56:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.766+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4e680696e0 con 0x7f4e6813b2c0 2024-01-19T04:56:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.906+0000 7f4e5dffb640 1 -- 172.21.15.31:0/647972938 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4e5800f040 con 0x7f4e6813b2c0 2024-01-19T04:56:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e4c06aed0 msgr2=0x7f4e4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e4c06aed0 0x7f4e4c06d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e64001c90 tx=0x7f4e64001b40 comp rx=0 tx=0).stop 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6813b2c0 msgr2=0x7f4e680af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6813b2c0 0x7f4e680af0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e5802bdc0 tx=0x7f4e58004730 comp rx=0 tx=0).stop 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 shutdown_connections 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e4c06aed0 0x7f4e4c06d390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e680af600 0x7f4e680ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6813b2c0 0x7f4e680af0c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 --2- 172.21.15.31:0/647972938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e540047c0 0x7f4e6813a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 >> 172.21.15.31:0/647972938 conn(0x7f4e6806a790 msgr2=0x7f4e680cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 shutdown_connections 2024-01-19T04:56:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:23.908+0000 7f4e6f7b0640 1 -- 172.21.15.31:0/647972938 wait complete. 2024-01-19T04:56:23.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-19T04:56:24.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.029+0000 7efcb3733640 1 Processor -- start 2024-01-19T04:56:24.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.029+0000 7efcb3733640 1 -- start start 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.030+0000 7efcb3733640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 0x7efcac0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.030+0000 7efcb3733640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac15ecb0 con 0x7efcac143440 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.030+0000 7efcb3733640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac15e130 con 0x7efcac06ddc0 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.030+0000 7efcb3733640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac072440 con 0x7efcac0b33f0 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.030+0000 7efcb14cf640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efcac143440 0x7efcac06d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39604/0 (socket says 172.21.15.31:39604) 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.030+0000 7efcb14cf640 1 -- 172.21.15.31:0/1138056509 learned_addr learned my addr 172.21.15.31:0/1138056509 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.030+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 0x7efcac0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129473131 0 0) 0x7efcac15ecb0 con 0x7efcac143440 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc98005730 con 0x7efcac143440 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644590377 0 0) 0x7efcac15e130 con 0x7efcac06ddc0 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcac15ecb0 con 0x7efcac06ddc0 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383059374 0 0) 0x7efc98005730 con 0x7efcac143440 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 msgr2=0x7efcac0cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 0x7efcac0cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 >> v1:172.21.15.137:6789/0 conn(0x7efcac06ddc0 legacy=0x7efcac0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac15e410 con 0x7efcac143440 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7efca4001670 con 0x7efcac143440 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 >> v1:172.21.15.31:6789/0 conn(0x7efcac143440 legacy=0x7efcac06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 0x7efc98005d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc980062a0 0x7efc980069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc98006f10 0x7efc98007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac072440 con 0x7efc98006f10 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9800dd20 con 0x7efc980062a0 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9800deb0 con 0x7efcac0b33f0 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc98006f10 0x7efc98007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb14cf640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 0x7efc98005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb0cce640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc980062a0 0x7efc980069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb1cd0640 1 -- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 msgr2=0x7efc98005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0b33f0 0x7efc98005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb1cd0640 1 -- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc980062a0 msgr2=0x7efc980069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc980062a0 0x7efc980069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.031+0000 7efcb1cd0640 1 -- 172.21.15.31:0/1138056509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc98005730 con 0x7efc98006f10 2024-01-19T04:56:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.032+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc98006f10 0x7efc98007390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efca8009f90 tx=0x7efc98007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799189051d313109 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.032+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca802cbd0 con 0x7efc98006f10 2024-01-19T04:56:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.032+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca800f040 con 0x7efc98006f10 2024-01-19T04:56:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.032+0000 7efca3fff640 1 -- 172.21.15.31:0/1138056509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8034440 con 0x7efc98006f10 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.033+0000 7efcb3733640 1 -- 172.21.15.31:0/1138056509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc98006f10 msgr2=0x7efc98007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.033+0000 7efcb3733640 1 --2- 172.21.15.31:0/1138056509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc98006f10 0x7efc98007390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efca8009f90 tx=0x7efc98007a00 comp rx=0 tx=0).stop 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.033+0000 7efcb1cd0640 1 -- 172.21.15.31:0/1138056509 reap_dead start 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.033+0000 7efcb3733640 1 -- 172.21.15.31:0/1138056509 shutdown_connections 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.033+0000 7efcb3733640 1 -- 172.21.15.31:0/1138056509 >> 172.21.15.31:0/1138056509 conn(0x7efcac06aa00 msgr2=0x7efcac06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.033+0000 7efcb3733640 1 -- 172.21.15.31:0/1138056509 shutdown_connections 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.033+0000 7efcb3733640 1 -- 172.21.15.31:0/1138056509 wait complete. 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.034+0000 7efcb3733640 1 Processor -- start 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.034+0000 7efcb3733640 1 -- start start 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb3733640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0cd9e0 0x7efcac15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb3733640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcac0cddc0 0x7efcac15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb3733640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcac0afa20 0x7efcac157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb3733640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac15c140 con 0x7efcac0afa20 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb3733640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac15c420 con 0x7efcac0cddc0 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb3733640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac15c700 con 0x7efcac0cd9e0 2024-01-19T04:56:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcac0afa20 0x7efcac157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcac0afa20 0x7efcac157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58108/0 (socket says 172.21.15.31:58108) 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 -- 172.21.15.31:0/3131950042 learned_addr learned my addr 172.21.15.31:0/3131950042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb14cf640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0cd9e0 0x7efcac15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb0cce640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcac0cddc0 0x7efcac15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 -- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0cd9e0 msgr2=0x7efcac15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0cd9e0 0x7efcac15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 -- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcac0cddc0 msgr2=0x7efcac15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcac0cddc0 0x7efcac15ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 -- 172.21.15.31:0/3131950042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac15c9e0 con 0x7efcac0afa20 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb0cce640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcac0cddc0 0x7efcac15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb14cf640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0cd9e0 0x7efcac15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.035+0000 7efcb1cd0640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcac0afa20 0x7efcac157750 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efca8037040 tx=0x7efca8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.036+0000 7efca1ffb640 1 -- 172.21.15.31:0/3131950042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8043070 con 0x7efcac0afa20 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.036+0000 7efca1ffb640 1 -- 172.21.15.31:0/3131950042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca802de60 con 0x7efcac0afa20 2024-01-19T04:56:24.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.036+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac15ccc0 con 0x7efcac0afa20 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.036+0000 7efca1ffb640 1 -- 172.21.15.31:0/3131950042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8034d90 con 0x7efcac0afa20 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.036+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac157c90 con 0x7efcac0afa20 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.037+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac074900 con 0x7efcac0afa20 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.037+0000 7efca1ffb640 1 -- 172.21.15.31:0/3131950042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efca803e440 con 0x7efcac0afa20 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.038+0000 7efca1ffb640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efc9006af00 0x7efc9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.038+0000 7efca1ffb640 1 -- 172.21.15.31:0/3131950042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7efca80aaff0 con 0x7efcac0afa20 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.038+0000 7efcb14cf640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efc9006af00 0x7efc9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.039+0000 7efcb14cf640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efc9006af00 0x7efc9006d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efcac0cc230 tx=0x7efca4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.042+0000 7efca1ffb640 1 -- 172.21.15.31:0/3131950042 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efca80760f0 con 0x7efcac0afa20 2024-01-19T04:56:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.221+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efcac0b31d0 con 0x7efcac0afa20 2024-01-19T04:56:24.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.908+0000 7efca1ffb640 1 -- 172.21.15.31:0/3131950042 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7efca8079710 con 0x7efcac0afa20 2024-01-19T04:56:24.908 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-01-19T04:56:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.911+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efc9006af00 msgr2=0x7efc9006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.911+0000 7efcb3733640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efc9006af00 0x7efc9006d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efcac0cc230 tx=0x7efca4009040 comp rx=0 tx=0).stop 2024-01-19T04:56:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.911+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcac0afa20 msgr2=0x7efcac157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.911+0000 7efcb3733640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcac0afa20 0x7efcac157750 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efca8037040 tx=0x7efca8009750 comp rx=0 tx=0).stop 2024-01-19T04:56:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.912+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 shutdown_connections 2024-01-19T04:56:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.912+0000 7efcb3733640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efc9006af00 0x7efc9006d3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.912+0000 7efcb3733640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcac0afa20 0x7efcac157750 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.912+0000 7efcb3733640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcac0cddc0 0x7efcac15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.912+0000 7efcb3733640 1 --2- 172.21.15.31:0/3131950042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efcac0cd9e0 0x7efcac15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.912+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 >> 172.21.15.31:0/3131950042 conn(0x7efcac06aa00 msgr2=0x7efcac06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.913+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 shutdown_connections 2024-01-19T04:56:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:24.913+0000 7efcb3733640 1 -- 172.21.15.31:0/3131950042 wait complete. 2024-01-19T04:56:24.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-19T04:56:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.048+0000 7f977b99e640 1 Processor -- start 2024-01-19T04:56:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.048+0000 7f977b99e640 1 -- start start 2024-01-19T04:56:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f977b99e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f97740cb2e0 0x7f977415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f977b99e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977415ece0 con 0x7f977406d6b0 2024-01-19T04:56:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f977b99e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9774071c20 con 0x7f97740716f0 2024-01-19T04:56:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f977b99e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9774071f00 con 0x7f97740cb2e0 2024-01-19T04:56:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f9779f3b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f977406d6b0 0x7f97740b1e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39614/0 (socket says 172.21.15.31:39614) 2024-01-19T04:56:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f9779f3b640 1 -- 172.21.15.31:0/240271572 learned_addr learned my addr 172.21.15.31:0/240271572 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f977973a640 1 --2- 172.21.15.31:0/240271572 >> v2:172.21.15.137:3300/0 conn(0x7f97740cb2e0 0x7f977415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444527922 0 0) 0x7f977415ece0 con 0x7f977406d6b0 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9760005730 con 0x7f977406d6b0 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2031946294 0 0) 0x7f9760005730 con 0x7f977406d6b0 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 >> v2:172.21.15.137:3300/0 conn(0x7f97740cb2e0 msgr2=0x7f977415e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 --2- 172.21.15.31:0/240271572 >> v2:172.21.15.137:3300/0 conn(0x7f97740cb2e0 0x7f977415e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 >> v1:172.21.15.137:6789/0 conn(0x7f97740716f0 legacy=0x7f977415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740721e0 con 0x7f977406d6b0 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f976c0019f0 con 0x7f977406d6b0 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 >> v1:172.21.15.31:6789/0 conn(0x7f977406d6b0 legacy=0x7f97740b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9760005b80 0x7f9760005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.049+0000 7f976bfff640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9760006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f976bfff640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9760007390 0x7f976000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9774071f00 con 0x7f97600069d0 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976000e230 con 0x7f9760005b80 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976000e3f0 con 0x7f9760007390 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9778f39640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9760006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f977973a640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9760005b80 0x7f9760005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9779f3b640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9760007390 0x7f976000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9778f39640 1 -- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9760007390 msgr2=0x7f976000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9778f39640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9760007390 0x7f976000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9778f39640 1 -- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9760005b80 msgr2=0x7f9760005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9778f39640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9760005b80 0x7f9760005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9778f39640 1 -- 172.21.15.31:0/240271572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9760005730 con 0x7f97600069d0 2024-01-19T04:56:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.050+0000 7f9779f3b640 1 -- 172.21.15.31:0/240271572 reap_dead start 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f9778f39640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9760006e50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9764001750 tx=0x7f9760007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a924c30eb6c117 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97640052d0 con 0x7f97600069d0 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976402e050 con 0x7f97600069d0 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f976bfff640 1 -- 172.21.15.31:0/240271572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764003830 con 0x7f97600069d0 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f977b99e640 1 -- 172.21.15.31:0/240271572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 msgr2=0x7f9760006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f977b99e640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9760006e50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9764001750 tx=0x7f9760007a00 comp rx=0 tx=0).stop 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f977b99e640 1 -- 172.21.15.31:0/240271572 shutdown_connections 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f977b99e640 1 --2- 172.21.15.31:0/240271572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9760006e50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f977b99e640 1 -- 172.21.15.31:0/240271572 >> 172.21.15.31:0/240271572 conn(0x7f977406aa00 msgr2=0x7f977406ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f977b99e640 1 -- 172.21.15.31:0/240271572 shutdown_connections 2024-01-19T04:56:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.051+0000 7f977b99e640 1 -- 172.21.15.31:0/240271572 wait complete. 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 Processor -- start 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 -- start start 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9774145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9774155e00 0x7f9774153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9774156240 0x7f9774154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9774157c20 con 0x7f97600069d0 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9774154e30 con 0x7f9774155e00 2024-01-19T04:56:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977b99e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9774155970 con 0x7f9774156240 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977973a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9774145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977973a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9774145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58130/0 (socket says 172.21.15.31:58130) 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.053+0000 7f977973a640 1 -- 172.21.15.31:0/2285258649 learned_addr learned my addr 172.21.15.31:0/2285258649 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f9778f39640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9774155e00 0x7f9774153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f9779f3b640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9774156240 0x7f9774154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977973a640 1 -- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9774156240 msgr2=0x7f9774154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977973a640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9774156240 0x7f9774154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977973a640 1 -- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9774155e00 msgr2=0x7f9774153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977973a640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9774155e00 0x7f9774153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977973a640 1 -- 172.21.15.31:0/2285258649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774155040 con 0x7f97600069d0 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977973a640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9774145780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9770011960 tx=0x7f9770011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f9769ffb640 1 -- 172.21.15.31:0/2285258649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000faf0 con 0x7f97600069d0 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f9769ffb640 1 -- 172.21.15.31:0/2285258649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977000fc50 con 0x7f97600069d0 2024-01-19T04:56:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9774155320 con 0x7f97600069d0 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f9769ffb640 1 -- 172.21.15.31:0/2285258649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770014440 con 0x7f97600069d0 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.054+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9774145cc0 con 0x7f97600069d0 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.055+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9774074900 con 0x7f97600069d0 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.056+0000 7f9769ffb640 1 -- 172.21.15.31:0/2285258649 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f977001d400 con 0x7f97600069d0 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.056+0000 7f9769ffb640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f975806aed0 0x7f975806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.056+0000 7f9769ffb640 1 -- 172.21.15.31:0/2285258649 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9770089eb0 con 0x7f97600069d0 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.056+0000 7f9778f39640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f975806aed0 0x7f975806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.057+0000 7f9778f39640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f975806aed0 0x7f975806d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9764033000 tx=0x7f9764009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:25.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.061+0000 7f9769ffb640 1 -- 172.21.15.31:0/2285258649 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9770054de0 con 0x7f97600069d0 2024-01-19T04:56:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.276+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f97740b31d0 con 0x7f97600069d0 2024-01-19T04:56:25.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.928+0000 7f9769ffb640 1 -- 172.21.15.31:0/2285258649 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9770058400 con 0x7f97600069d0 2024-01-19T04:56:25.928 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 4 and data pool 5 2024-01-19T04:56:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.937+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f975806aed0 msgr2=0x7f975806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.937+0000 7f977b99e640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f975806aed0 0x7f975806d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9764033000 tx=0x7f9764009030 comp rx=0 tx=0).stop 2024-01-19T04:56:25.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.937+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 msgr2=0x7f9774145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:25.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.937+0000 7f977b99e640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9774145780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9770011960 tx=0x7f9770011e20 comp rx=0 tx=0).stop 2024-01-19T04:56:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.938+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 shutdown_connections 2024-01-19T04:56:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.938+0000 7f977b99e640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f975806aed0 0x7f975806d390 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9764033000 tx=0x7f9764009030 comp rx=0 tx=0).stop 2024-01-19T04:56:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.938+0000 7f977b99e640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9774156240 0x7f9774154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.938+0000 7f977b99e640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9774155e00 0x7f9774153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.938+0000 7f977b99e640 1 --2- 172.21.15.31:0/2285258649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97600069d0 0x7f9774145780 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.938+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 >> 172.21.15.31:0/2285258649 conn(0x7f977406aa00 msgr2=0x7f9774070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.942+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 shutdown_connections 2024-01-19T04:56:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:25.942+0000 7f977b99e640 1 -- 172.21.15.31:0/2285258649 wait complete. 2024-01-19T04:56:25.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.063+0000 7f1183b9e640 1 Processor -- start 2024-01-19T04:56:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.063+0000 7f1183b9e640 1 -- start start 2024-01-19T04:56:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f1183b9e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f1183b9e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f117c0cba90 0x7f117c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f1183b9e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f117c06b500 0x7f117c0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f1183b9e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c06c970 con 0x7f117c06b500 2024-01-19T04:56:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f1183b9e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c06cf30 con 0x7f117c0cba90 2024-01-19T04:56:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f1183b9e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c06f120 con 0x7f117c0ca310 2024-01-19T04:56:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f117c06b500 0x7f117c0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f117c06b500 0x7f117c0ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58144/0 (socket says 172.21.15.31:58144) 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 -- 172.21.15.31:0/826085951 learned_addr learned my addr 172.21.15.31:0/826085951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f1181139640 1 --2- 172.21.15.31:0/826085951 >> v2:172.21.15.137:3300/0 conn(0x7f117c0cba90 0x7f117c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118193a640 1 --2- 172.21.15.31:0/826085951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 -- 172.21.15.31:0/826085951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 msgr2=0x7f117c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 --2- 172.21.15.31:0/826085951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 -- 172.21.15.31:0/826085951 >> v2:172.21.15.137:3300/0 conn(0x7f117c0cba90 msgr2=0x7f117c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 --2- 172.21.15.31:0/826085951 >> v2:172.21.15.137:3300/0 conn(0x7f117c0cba90 0x7f117c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.064+0000 7f118213b640 1 -- 172.21.15.31:0/826085951 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c06f400 con 0x7f117c06b500 2024-01-19T04:56:26.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.065+0000 7f118213b640 1 --2- 172.21.15.31:0/826085951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c06b500 0x7f117c0ca6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f117c0cf620 tx=0x7f117402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36fdbd33bd5dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.065+0000 7f1180938640 1 -- 172.21.15.31:0/826085951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117402ca70 con 0x7f117c06b500 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.065+0000 7f1180938640 1 -- 172.21.15.31:0/826085951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117402cbd0 con 0x7f117c06b500 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.065+0000 7f1180938640 1 -- 172.21.15.31:0/826085951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740343f0 con 0x7f117c06b500 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 -- 172.21.15.31:0/826085951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c06b500 msgr2=0x7f117c0ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 --2- 172.21.15.31:0/826085951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c06b500 0x7f117c0ca6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f117c0cf620 tx=0x7f117402bed0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 -- 172.21.15.31:0/826085951 shutdown_connections 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 --2- 172.21.15.31:0/826085951 >> v2:172.21.15.137:3300/0 conn(0x7f117c0cba90 0x7f117c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 --2- 172.21.15.31:0/826085951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c06b500 0x7f117c0ca6f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 --2- 172.21.15.31:0/826085951 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 -- 172.21.15.31:0/826085951 >> 172.21.15.31:0/826085951 conn(0x7f117c0cc1b0 msgr2=0x7f117c0cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 -- 172.21.15.31:0/826085951 shutdown_connections 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.066+0000 7f1183b9e640 1 -- 172.21.15.31:0/826085951 wait complete. 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.067+0000 7f1183b9e640 1 Processor -- start 2024-01-19T04:56:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.067+0000 7f1183b9e640 1 -- start start 2024-01-19T04:56:26.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f1183b9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c06b500 0x7f117c1481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f1183b9e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f1183b9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c0cba90 0x7f117c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f1183b9e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c1550d0 con 0x7f117c0cba90 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f1183b9e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c148700 con 0x7f117c06b500 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f1183b9e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c154df0 con 0x7f117c0ca310 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c0cba90 0x7f117c1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c0cba90 0x7f117c1524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58158/0 (socket says 172.21.15.31:58158) 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 -- 172.21.15.31:0/2884663774 learned_addr learned my addr 172.21.15.31:0/2884663774 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f1181139640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118193a640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c06b500 0x7f117c1481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 -- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 msgr2=0x7f117c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 -- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c06b500 msgr2=0x7f117c1481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c06b500 0x7f117c1481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 -- 172.21.15.31:0/2884663774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174009b30 con 0x7f117c0cba90 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.068+0000 7f118213b640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c0cba90 0x7f117c1524c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1174036040 tx=0x7f1174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.069+0000 7f11727fc640 1 -- 172.21.15.31:0/2884663774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174009510 con 0x7f117c0cba90 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.069+0000 7f11727fc640 1 -- 172.21.15.31:0/2884663774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117403e450 con 0x7f117c0cba90 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.069+0000 7f11727fc640 1 -- 172.21.15.31:0/2884663774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117403e800 con 0x7f117c0cba90 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.069+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c15c980 con 0x7f117c0cba90 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.069+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c15cf40 con 0x7f117c0cba90 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.070+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c0cd150 con 0x7f117c0cba90 2024-01-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.070+0000 7f11727fc640 1 -- 172.21.15.31:0/2884663774 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f117403e960 con 0x7f117c0cba90 2024-01-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.070+0000 7f11727fc640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f116006aed0 0x7f116006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.071+0000 7f11727fc640 1 -- 172.21.15.31:0/2884663774 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11740ac1d0 con 0x7f117c0cba90 2024-01-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.071+0000 7f118193a640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f116006aed0 0x7f116006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.071+0000 7f118193a640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f116006aed0 0x7f116006d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1178009ac0 tx=0x7f1178009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.075+0000 7f11727fc640 1 -- 172.21.15.31:0/2884663774 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f11740770c0 con 0x7f117c0cba90 2024-01-19T04:56:26.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.254+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f117c15d160 con 0x7f117c0cba90 2024-01-19T04:56:26.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.256+0000 7f11727fc640 1 -- 172.21.15.31:0/2884663774 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f117407a6e0 con 0x7f117c0cba90 2024-01-19T04:56:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:26.255 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:25.277897+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:26.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f116006aed0 msgr2=0x7f116006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f116006aed0 0x7f116006d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1178009ac0 tx=0x7f1178009190 comp rx=0 tx=0).stop 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c0cba90 msgr2=0x7f117c1524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c0cba90 0x7f117c1524c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1174036040 tx=0x7f1174004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 shutdown_connections 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f116006aed0 0x7f116006d390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c0cba90 0x7f117c1524c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f117c0ca310 0x7f117c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 --2- 172.21.15.31:0/2884663774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c06b500 0x7f117c1481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 >> 172.21.15.31:0/2884663774 conn(0x7f117c0cc1b0 msgr2=0x7f117c0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 shutdown_connections 2024-01-19T04:56:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.258+0000 7f1183b9e640 1 -- 172.21.15.31:0/2884663774 wait complete. 2024-01-19T04:56:26.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:26.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.369+0000 7fe99e68e640 1 Processor -- start 2024-01-19T04:56:26.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.369+0000 7fe99e68e640 1 -- start start 2024-01-19T04:56:26.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.370+0000 7fe99e68e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe9980d2e50 0x7fe9980b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.370+0000 7fe99e68e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980b4d00 0x7fe9980b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.370+0000 7fe99e68e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe998160df0 con 0x7fe998160a10 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.370+0000 7fe99e68e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9980b7300 con 0x7fe9980d2e50 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.370+0000 7fe99e68e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980b5680 con 0x7fe9980b4d00 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe99cc2b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980b4d00 0x7fe9980b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe9977fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe9980d2e50 0x7fe9980b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe9977fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe9980d2e50 0x7fe9980b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:53626/0 (socket says 172.21.15.31:53626) 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe9977fe640 1 -- 172.21.15.31:0/1360216526 learned_addr learned my addr 172.21.15.31:0/1360216526 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe99cc2b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980b4d00 0x7fe9980b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:38992/0 (socket says 172.21.15.31:38992) 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe9977fe640 1 -- 172.21.15.31:0/1360216526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980b4d00 msgr2=0x7fe9980b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe9977fe640 1 --2- 172.21.15.31:0/1360216526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980b4d00 0x7fe9980b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe9977fe640 1 -- 172.21.15.31:0/1360216526 >> v1:172.21.15.137:6789/0 conn(0x7fe998160a10 legacy=0x7fe9980d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.371+0000 7fe9977fe640 1 -- 172.21.15.31:0/1360216526 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980d1db0 con 0x7fe9980d2e50 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe9977fe640 1 --2- 172.21.15.31:0/1360216526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980d2e50 0x7fe9980b8ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe980009870 tx=0x7fe98002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e770ad718f577a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe996ffd640 1 -- 172.21.15.31:0/1360216526 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98002f840 con 0x7fe9980d2e50 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe996ffd640 1 -- 172.21.15.31:0/1360216526 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98002f9a0 con 0x7fe9980d2e50 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe996ffd640 1 -- 172.21.15.31:0/1360216526 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98002fd30 con 0x7fe9980d2e50 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 -- 172.21.15.31:0/1360216526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980d2e50 msgr2=0x7fe9980b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1360216526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980d2e50 0x7fe9980b8ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe980009870 tx=0x7fe98002ba20 comp rx=0 tx=0).stop 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 -- 172.21.15.31:0/1360216526 shutdown_connections 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1360216526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980b4d00 0x7fe9980b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1360216526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980d2e50 0x7fe9980b8ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 -- 172.21.15.31:0/1360216526 >> 172.21.15.31:0/1360216526 conn(0x7fe998072310 msgr2=0x7fe9980cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:26.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 -- 172.21.15.31:0/1360216526 shutdown_connections 2024-01-19T04:56:26.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.372+0000 7fe99e68e640 1 -- 172.21.15.31:0/1360216526 wait complete. 2024-01-19T04:56:26.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 Processor -- start 2024-01-19T04:56:26.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 -- start start 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980b4d00 0x7fe9980c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980d2e50 0x7fe99814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998160a10 0x7fe9980cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99814bbe0 con 0x7fe998160a10 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980c7da0 con 0x7fe9980b4d00 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.374+0000 7fe99e68e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980c8080 con 0x7fe9980d2e50 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998160a10 0x7fe9980cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998160a10 0x7fe9980cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58164/0 (socket says 172.21.15.31:58164) 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 -- 172.21.15.31:0/1872666185 learned_addr learned my addr 172.21.15.31:0/1872666185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe9977fe640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980d2e50 0x7fe99814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe997fff640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980b4d00 0x7fe9980c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 -- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980d2e50 msgr2=0x7fe99814b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980d2e50 0x7fe99814b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 -- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980b4d00 msgr2=0x7fe9980c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980b4d00 0x7fe9980c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 -- 172.21.15.31:0/1872666185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9800094d0 con 0x7fe998160a10 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99cc2b640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998160a10 0x7fe9980cc210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe99000efc0 tx=0x7fe99000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe994ff9640 1 -- 172.21.15.31:0/1872666185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990009ba0 con 0x7fe998160a10 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe994ff9640 1 -- 172.21.15.31:0/1872666185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99000f040 con 0x7fe998160a10 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe994ff9640 1 -- 172.21.15.31:0/1872666185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900146c0 con 0x7fe998160a10 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9980c8360 con 0x7fe998160a10 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.375+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9980cc750 con 0x7fe998160a10 2024-01-19T04:56:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.376+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9980696e0 con 0x7fe998160a10 2024-01-19T04:56:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.377+0000 7fe994ff9640 1 -- 172.21.15.31:0/1872666185 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe990014820 con 0x7fe998160a10 2024-01-19T04:56:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.377+0000 7fe994ff9640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe97806aed0 0x7fe97806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.377+0000 7fe994ff9640 1 -- 172.21.15.31:0/1872666185 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe99008a030 con 0x7fe998160a10 2024-01-19T04:56:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.377+0000 7fe997fff640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe97806aed0 0x7fe97806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.378+0000 7fe997fff640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe97806aed0 0x7fe97806d390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe9980b61f0 tx=0x7fe988009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.382+0000 7fe994ff9640 1 -- 172.21.15.31:0/1872666185 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe990054fb0 con 0x7fe998160a10 2024-01-19T04:56:26.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.559+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe99806a5b0 con 0x7fe998160a10 2024-01-19T04:56:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.560+0000 7fe994ff9640 1 -- 172.21.15.31:0/1872666185 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16547 (secure 0 0 0) 0x7fe9900585d0 con 0x7fe998160a10 2024-01-19T04:56:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:26.560 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:25.277897+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe97806aed0 msgr2=0x7fe97806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe97806aed0 0x7fe97806d390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe9980b61f0 tx=0x7fe988009040 comp rx=0 tx=0).stop 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998160a10 msgr2=0x7fe9980cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998160a10 0x7fe9980cc210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe99000efc0 tx=0x7fe99000c490 comp rx=0 tx=0).stop 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 shutdown_connections 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe97806aed0 0x7fe97806d390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998160a10 0x7fe9980cc210 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9980d2e50 0x7fe99814b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 --2- 172.21.15.31:0/1872666185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9980b4d00 0x7fe9980c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 >> 172.21.15.31:0/1872666185 conn(0x7fe998072310 msgr2=0x7fe9980ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 shutdown_connections 2024-01-19T04:56:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.562+0000 7fe99e68e640 1 -- 172.21.15.31:0/1872666185 wait complete. 2024-01-19T04:56:26.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-19T04:56:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.677+0000 7f19b7f94640 1 Processor -- start 2024-01-19T04:56:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.677+0000 7f19b7f94640 1 -- start start 2024-01-19T04:56:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.678+0000 7f19b7f94640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 0x7f19b00d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.678+0000 7f19b7f94640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b00cf160 con 0x7f19b00d1c20 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.678+0000 7f19b7f94640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b00ce8c0 con 0x7f19b015d720 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.678+0000 7f19b7f94640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00ceba0 con 0x7f19b00d3ea0 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.678+0000 7f19b552f640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f19b00d1c20 0x7f19b00b3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39618/0 (socket says 172.21.15.31:39618) 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.678+0000 7f19b552f640 1 -- 172.21.15.31:0/4085766328 learned_addr learned my addr 172.21.15.31:0/4085766328 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436400473 0 0) 0x7f19b00cf160 con 0x7f19b00d1c20 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1998003520 con 0x7f19b00d1c20 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b6531640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 0x7f19b00d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 854932027 0 0) 0x7f1998003520 con 0x7f19b00d1c20 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 msgr2=0x7f19b00d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 0x7f19b00d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 >> v1:172.21.15.137:6789/0 conn(0x7f19b015d720 legacy=0x7f19b00d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00cee80 con 0x7f19b00d1c20 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f19a4001670 con 0x7f19b00d1c20 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 >> v1:172.21.15.31:6789/0 conn(0x7f19b00d1c20 legacy=0x7f19b00b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19980039a0 0x7f1998003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 0x7f19980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1998004d00 0x7f1998005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00ceba0 con 0x7f1998004d00 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199800bb40 con 0x7f19980039a0 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199800bd00 con 0x7f19b00d3ea0 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b6531640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1998004d00 0x7f1998005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b5d30640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19980039a0 0x7f1998003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b552f640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 0x7f19980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b6531640 1 -- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 msgr2=0x7f19980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b6531640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b00d3ea0 0x7f19980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b6531640 1 -- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19980039a0 msgr2=0x7f1998003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b6531640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19980039a0 0x7f1998003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.679+0000 7f19b6531640 1 -- 172.21.15.31:0/4085766328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998003520 con 0x7f1998004d00 2024-01-19T04:56:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.680+0000 7f19b6531640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1998004d00 0x7f1998005180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009a90 tx=0x7f19980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244003a33c5e1e99 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.680+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a802c680 con 0x7f1998004d00 2024-01-19T04:56:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.680+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a802c7e0 con 0x7f1998004d00 2024-01-19T04:56:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.680+0000 7f19b4d2e640 1 -- 172.21.15.31:0/4085766328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a802cb90 con 0x7f1998004d00 2024-01-19T04:56:26.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.681+0000 7f19b7f94640 1 -- 172.21.15.31:0/4085766328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1998004d00 msgr2=0x7f1998005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.681+0000 7f19b7f94640 1 --2- 172.21.15.31:0/4085766328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1998004d00 0x7f1998005180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009a90 tx=0x7f19980057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.681+0000 7f19b6531640 1 -- 172.21.15.31:0/4085766328 reap_dead start 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.681+0000 7f19b7f94640 1 -- 172.21.15.31:0/4085766328 shutdown_connections 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.681+0000 7f19b7f94640 1 -- 172.21.15.31:0/4085766328 >> 172.21.15.31:0/4085766328 conn(0x7f19b00d3310 msgr2=0x7f19b00d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.681+0000 7f19b7f94640 1 -- 172.21.15.31:0/4085766328 shutdown_connections 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.681+0000 7f19b7f94640 1 -- 172.21.15.31:0/4085766328 wait complete. 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.682+0000 7f19b7f94640 1 Processor -- start 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.682+0000 7f19b7f94640 1 -- start start 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b0155960 0x7f19b00c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b0155d40 0x7f19b00c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19b00c8450 0x7f19b015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0153310 con 0x7f19b0155960 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b015f190 con 0x7f19b00c8450 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b015f470 con 0x7f19b0155d40 2024-01-19T04:56:26.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b0155960 0x7f19b00c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b0155960 0x7f19b00c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58192/0 (socket says 172.21.15.31:58192) 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 -- 172.21.15.31:0/2894760663 learned_addr learned my addr 172.21.15.31:0/2894760663 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b552f640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b0155d40 0x7f19b00c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b6531640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19b00c8450 0x7f19b015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 -- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b0155d40 msgr2=0x7f19b00c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b0155d40 0x7f19b00c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 -- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19b00c8450 msgr2=0x7f19b015dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19b00c8450 0x7f19b015dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 -- 172.21.15.31:0/2894760663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b0153560 con 0x7f19b0155960 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b6531640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19b00c8450 0x7f19b015dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b5d30640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b0155960 0x7f19b00c8d70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fb90 tx=0x7f19a000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19ae7fc640 1 -- 172.21.15.31:0/2894760663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000fe90 con 0x7f19b0155960 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19ae7fc640 1 -- 172.21.15.31:0/2894760663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a000dc30 con 0x7f19b0155960 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b0153840 con 0x7f19b0155960 2024-01-19T04:56:26.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19ae7fc640 1 -- 172.21.15.31:0/2894760663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0003970 con 0x7f19b0155960 2024-01-19T04:56:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.683+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b00c92b0 con 0x7f19b0155960 2024-01-19T04:56:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.684+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b015f630 con 0x7f19b0155960 2024-01-19T04:56:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.685+0000 7f19ae7fc640 1 -- 172.21.15.31:0/2894760663 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f19a0003ad0 con 0x7f19b0155960 2024-01-19T04:56:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.685+0000 7f19ae7fc640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f199c06af00 0x7f199c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.685+0000 7f19ae7fc640 1 -- 172.21.15.31:0/2894760663 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19a0089970 con 0x7f19b0155960 2024-01-19T04:56:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.685+0000 7f19b552f640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f199c06af00 0x7f199c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.686+0000 7f19b552f640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f199c06af00 0x7f199c06d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f19a4004840 tx=0x7f19a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.690+0000 7f19ae7fc640 1 -- 172.21.15.31:0/2894760663 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f19a00547c0 con 0x7f19b0155960 2024-01-19T04:56:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.905+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f19b01539d0 con 0x7f19b0155960 2024-01-19T04:56:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.929+0000 7f19ae7fc640 1 -- 172.21.15.31:0/2894760663 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7f19a0057de0 con 0x7f19b0155960 2024-01-19T04:56:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f199c06af00 msgr2=0x7f199c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f199c06af00 0x7f199c06d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f19a4004840 tx=0x7f19a4009040 comp rx=0 tx=0).stop 2024-01-19T04:56:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b0155960 msgr2=0x7f19b00c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b0155960 0x7f19b00c8d70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fb90 tx=0x7f19a000d520 comp rx=0 tx=0).stop 2024-01-19T04:56:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 shutdown_connections 2024-01-19T04:56:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f199c06af00 0x7f199c06d3c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19b00c8450 0x7f19b015dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19b0155d40 0x7f19b00c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 --2- 172.21.15.31:0/2894760663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b0155960 0x7f19b00c8d70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:26.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 >> 172.21.15.31:0/2894760663 conn(0x7f19b00d3310 msgr2=0x7f19b006b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:26.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 shutdown_connections 2024-01-19T04:56:26.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:26.931+0000 7f19b7f94640 1 -- 172.21.15.31:0/2894760663 wait complete. 2024-01-19T04:56:26.943 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-19T04:56:26.943 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-19T04:56:27.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.047+0000 7f6e72f9e640 1 Processor -- start 2024-01-19T04:56:27.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.047+0000 7f6e72f9e640 1 -- start start 2024-01-19T04:56:27.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e72f9e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e6c0d18e0 0x7f6e6c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:27.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e72f9e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d0ef0 0x7f6e6c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:27.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e72f9e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c15dee0 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e72f9e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c15e1c0 con 0x7f6e6c0d18e0 2024-01-19T04:56:27.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e72f9e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15e4a0 con 0x7f6e6c0d0ef0 2024-01-19T04:56:27.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e70d3a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e6c0d18e0 0x7f6e6c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:27.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e70d3a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e6c0d18e0 0x7f6e6c15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58200/0 (socket says 172.21.15.31:58200) 2024-01-19T04:56:27.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e70d3a640 1 -- 172.21.15.31:0/3312467291 learned_addr learned my addr 172.21.15.31:0/3312467291 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:27.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e7153b640 1 --2- 172.21.15.31:0/3312467291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d0ef0 0x7f6e6c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:27.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e70d3a640 1 -- 172.21.15.31:0/3312467291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d0ef0 msgr2=0x7f6e6c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:27.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e70d3a640 1 --2- 172.21.15.31:0/3312467291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d0ef0 0x7f6e6c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e70d3a640 1 -- 172.21.15.31:0/3312467291 >> v1:172.21.15.137:6789/0 conn(0x7f6e6c0d3e50 legacy=0x7f6e6c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:27.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.048+0000 7f6e70d3a640 1 -- 172.21.15.31:0/3312467291 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c15e780 con 0x7f6e6c0d18e0 2024-01-19T04:56:27.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.049+0000 7f6e70d3a640 1 --2- 172.21.15.31:0/3312467291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d18e0 0x7f6e6c15d860 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe5a1bcda2632a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:27.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.049+0000 7f6e6b7fe640 1 -- 172.21.15.31:0/3312467291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6402ca70 con 0x7f6e6c0d18e0 2024-01-19T04:56:27.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.049+0000 7f6e6b7fe640 1 -- 172.21.15.31:0/3312467291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6402cbd0 con 0x7f6e6c0d18e0 2024-01-19T04:56:27.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.049+0000 7f6e6b7fe640 1 -- 172.21.15.31:0/3312467291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640343f0 con 0x7f6e6c0d18e0 2024-01-19T04:56:27.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3312467291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d18e0 msgr2=0x7f6e6c15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3312467291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d18e0 0x7f6e6c15d860 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6402bed0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3312467291 shutdown_connections 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3312467291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d0ef0 0x7f6e6c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3312467291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d18e0 0x7f6e6c15d860 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3312467291 >> 172.21.15.31:0/3312467291 conn(0x7f6e6c09c600 msgr2=0x7f6e6c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3312467291 shutdown_connections 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.050+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3312467291 wait complete. 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.051+0000 7f6e72f9e640 1 Processor -- start 2024-01-19T04:56:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.051+0000 7f6e72f9e640 1 -- start start 2024-01-19T04:56:27.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e72f9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e6c0d0ef0 0x7f6e6c150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:27.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e72f9e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d18e0 0x7f6e6c0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:27.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e72f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d3e50 0x7f6e6c0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:27.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e72f9e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0b8c50 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e72f9e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0c7c20 con 0x7f6e6c0d0ef0 2024-01-19T04:56:27.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e72f9e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0c7e90 con 0x7f6e6c0d18e0 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d3e50 0x7f6e6c0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d3e50 0x7f6e6c0b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58214/0 (socket says 172.21.15.31:58214) 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 -- 172.21.15.31:0/3504094844 learned_addr learned my addr 172.21.15.31:0/3504094844 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e70d3a640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e6c0d0ef0 0x7f6e6c150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e6bfff640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d18e0 0x7f6e6c0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 -- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d18e0 msgr2=0x7f6e6c0b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d18e0 0x7f6e6c0b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 -- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e6c0d0ef0 msgr2=0x7f6e6c150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e6c0d0ef0 0x7f6e6c150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.052+0000 7f6e7153b640 1 -- 172.21.15.31:0/3504094844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64009b30 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.053+0000 7f6e7153b640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d3e50 0x7f6e6c0b8710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00c820 tx=0x7f6e5c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.053+0000 7f6e697fa640 1 -- 172.21.15.31:0/3504094844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c004020 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.053+0000 7f6e697fa640 1 -- 172.21.15.31:0/3504094844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c004180 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.053+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c0c8170 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.053+0000 7f6e697fa640 1 -- 172.21.15.31:0/3504094844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c007830 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.053+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c0c8730 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.054+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c0c8950 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.054+0000 7f6e697fa640 1 -- 172.21.15.31:0/3504094844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6e5c007990 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.055+0000 7f6e697fa640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6e4c06aed0 0x7f6e4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:27.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.055+0000 7f6e697fa640 1 -- 172.21.15.31:0/3504094844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e5c089250 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.055+0000 7f6e70d3a640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6e4c06aed0 0x7f6e4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:27.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.056+0000 7f6e70d3a640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6e4c06aed0 0x7f6e4c06d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e64035000 tx=0x7f6e64030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.059+0000 7f6e697fa640 1 -- 172.21.15.31:0/3504094844 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6e5c054260 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.274+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6e6c074900 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.940+0000 7f6e697fa640 1 -- 172.21.15.31:0/3504094844 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6e5c057880 con 0x7f6e6c0d3e50 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6e4c06aed0 msgr2=0x7f6e4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6e4c06aed0 0x7f6e4c06d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e64035000 tx=0x7f6e64030000 comp rx=0 tx=0).stop 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d3e50 msgr2=0x7f6e6c0b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d3e50 0x7f6e6c0b8710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00c820 tx=0x7f6e5c00cce0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 shutdown_connections 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6e4c06aed0 0x7f6e4c06d390 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e6c0d3e50 0x7f6e6c0b8710 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6e6c0d18e0 0x7f6e6c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.943+0000 7f6e72f9e640 1 --2- 172.21.15.31:0/3504094844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e6c0d0ef0 0x7f6e6c150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.944+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 >> 172.21.15.31:0/3504094844 conn(0x7f6e6c09c600 msgr2=0x7f6e6c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.944+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 shutdown_connections 2024-01-19T04:56:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:27.944+0000 7f6e72f9e640 1 -- 172.21.15.31:0/3504094844 wait complete. 2024-01-19T04:56:27.956 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-19T04:56:27.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:28.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.064+0000 7f380890f640 1 Processor -- start 2024-01-19T04:56:28.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.064+0000 7f380890f640 1 -- start start 2024-01-19T04:56:28.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.065+0000 7f380890f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3804160a10 0x7f38040b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.065+0000 7f380890f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38040b4d00 0x7f38040b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.065+0000 7f380890f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3804160df0 con 0x7f38040d2e50 2024-01-19T04:56:28.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.065+0000 7f380890f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38040b7300 con 0x7f3804160a10 2024-01-19T04:56:28.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.065+0000 7f380890f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38040b5660 con 0x7f38040b4d00 2024-01-19T04:56:28.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f380259c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3804160a10 0x7f38040b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38040b4d00 0x7f38040b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f380259c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3804160a10 0x7f38040b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:53664/0 (socket says 172.21.15.31:53664) 2024-01-19T04:56:28.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38040b4d00 0x7f38040b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:39044/0 (socket says 172.21.15.31:39044) 2024-01-19T04:56:28.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f380259c640 1 -- 172.21.15.31:0/2320279402 learned_addr learned my addr 172.21.15.31:0/2320279402 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:28.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f3801d9b640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f38040d2e50 0x7f38040d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:55660/0 (socket says 172.21.15.31:55660) 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f3802d9d640 1 -- 172.21.15.31:0/2320279402 >> v2:172.21.15.137:3300/0 conn(0x7f3804160a10 msgr2=0x7f38040b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f3802d9d640 1 --2- 172.21.15.31:0/2320279402 >> v2:172.21.15.137:3300/0 conn(0x7f3804160a10 0x7f38040b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f3802d9d640 1 -- 172.21.15.31:0/2320279402 >> v1:172.21.15.137:6789/0 conn(0x7f38040d2e50 legacy=0x7f38040d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f3802d9d640 1 -- 172.21.15.31:0/2320279402 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040d1db0 con 0x7f38040b4d00 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f380259c640 1 --2- 172.21.15.31:0/2320279402 >> v2:172.21.15.137:3300/0 conn(0x7f3804160a10 0x7f38040b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.066+0000 7f380159a640 1 -- 172.21.15.31:0/2320279402 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038488007 0 0) 0x7f3804160df0 con 0x7f38040d2e50 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.067+0000 7f3802d9d640 1 --2- 172.21.15.31:0/2320279402 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38040b4d00 0x7f38040b5120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009d90 tx=0x7f37f402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d3bfd8fa61961dc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.067+0000 7f380159a640 1 -- 172.21.15.31:0/2320279402 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f402cce0 con 0x7f38040b4d00 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.067+0000 7f380159a640 1 -- 172.21.15.31:0/2320279402 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f402ce40 con 0x7f38040b4d00 2024-01-19T04:56:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.067+0000 7f380159a640 1 -- 172.21.15.31:0/2320279402 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4034660 con 0x7f38040b4d00 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 -- 172.21.15.31:0/2320279402 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38040b4d00 msgr2=0x7f38040b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 --2- 172.21.15.31:0/2320279402 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38040b4d00 0x7f38040b5120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009d90 tx=0x7f37f402bf30 comp rx=0 tx=0).stop 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 -- 172.21.15.31:0/2320279402 shutdown_connections 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 --2- 172.21.15.31:0/2320279402 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38040b4d00 0x7f38040b5120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 --2- 172.21.15.31:0/2320279402 >> v2:172.21.15.137:3300/0 conn(0x7f3804160a10 0x7f38040b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 -- 172.21.15.31:0/2320279402 >> 172.21.15.31:0/2320279402 conn(0x7f3804072310 msgr2=0x7f38040cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 -- 172.21.15.31:0/2320279402 shutdown_connections 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.068+0000 7f380890f640 1 -- 172.21.15.31:0/2320279402 wait complete. 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.069+0000 7f380890f640 1 Processor -- start 2024-01-19T04:56:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.069+0000 7f380890f640 1 -- start start 2024-01-19T04:56:28.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380890f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38040b4d00 0x7f380414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380890f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38040d2e50 0x7f380415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380890f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3804160a10 0x7f380415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380890f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380415ab40 con 0x7f38040b4d00 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380890f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380415ae20 con 0x7f38040d2e50 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380890f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380415b100 con 0x7f3804160a10 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38040b4d00 0x7f380414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38040b4d00 0x7f380414c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58228/0 (socket says 172.21.15.31:58228) 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f380259c640 1 -- 172.21.15.31:0/1807046877 learned_addr learned my addr 172.21.15.31:0/1807046877 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f3801d9b640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38040d2e50 0x7f380415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.070+0000 7f3802d9d640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3804160a10 0x7f380415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380259c640 1 -- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3804160a10 msgr2=0x7f380415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380259c640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3804160a10 0x7f380415f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380259c640 1 -- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38040d2e50 msgr2=0x7f380415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380259c640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38040d2e50 0x7f380415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380259c640 1 -- 172.21.15.31:0/1807046877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0094d0 con 0x7f38040b4d00 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f3801d9b640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38040d2e50 0x7f380415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f3802d9d640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3804160a10 0x7f380415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380259c640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38040b4d00 0x7f380414c600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00e9b0 tx=0x7f37fc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f37faffd640 1 -- 172.21.15.31:0/1807046877 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00c9a0 con 0x7f38040b4d00 2024-01-19T04:56:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f37faffd640 1 -- 172.21.15.31:0/1807046877 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc00cb00 con 0x7f38040b4d00 2024-01-19T04:56:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f4009a70 con 0x7f38040b4d00 2024-01-19T04:56:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f37faffd640 1 -- 172.21.15.31:0/1807046877 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00ce70 con 0x7f38040b4d00 2024-01-19T04:56:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.071+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38041577f0 con 0x7f38040b4d00 2024-01-19T04:56:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.072+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38040696e0 con 0x7f38040b4d00 2024-01-19T04:56:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.072+0000 7f37faffd640 1 -- 172.21.15.31:0/1807046877 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f37fc013470 con 0x7f38040b4d00 2024-01-19T04:56:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.073+0000 7f37faffd640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f37e406af00 0x7f37e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.073+0000 7f37faffd640 1 -- 172.21.15.31:0/1807046877 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37fc089b80 con 0x7f38040b4d00 2024-01-19T04:56:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.073+0000 7f3801d9b640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f37e406af00 0x7f37e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.074+0000 7f3801d9b640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f37e406af00 0x7f37e406d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38040cc560 tx=0x7f37ec007c10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.078+0000 7f37faffd640 1 -- 172.21.15.31:0/1807046877 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f37fc054ae0 con 0x7f38040b4d00 2024-01-19T04:56:28.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.292+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38040d0b50 con 0x7f38040b4d00 2024-01-19T04:56:28.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.293+0000 7f37faffd640 1 -- 172.21.15.31:0/1807046877 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f37fc017090 con 0x7f38040b4d00 2024-01-19T04:56:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:28.293 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:56:25.278275+0000","modified":"2024-01-19T04:56:27.276135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5259":{"gid":5259,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/968278604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":968278604},{"type":"v1","addr":"172.21.15.31:6835","nonce":968278604}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-19T04:56:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 14 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f37e406af00 msgr2=0x7f37e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f37e406af00 0x7f37e406d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38040cc560 tx=0x7f37ec007c10 comp rx=0 tx=0).stop 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38040b4d00 msgr2=0x7f380414c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38040b4d00 0x7f380414c600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00e9b0 tx=0x7f37fc00ee70 comp rx=0 tx=0).stop 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 shutdown_connections 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f37e406af00 0x7f37e406d3c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3804160a10 0x7f380415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38040d2e50 0x7f380415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 --2- 172.21.15.31:0/1807046877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38040b4d00 0x7f380414c600 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 >> 172.21.15.31:0/1807046877 conn(0x7f3804072310 msgr2=0x7f38040cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 shutdown_connections 2024-01-19T04:56:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.295+0000 7f380890f640 1 -- 172.21.15.31:0/1807046877 wait complete. 2024-01-19T04:56:28.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:28.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.406+0000 7f08fc55d640 1 Processor -- start 2024-01-19T04:56:28.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.406+0000 7f08fc55d640 1 -- start start 2024-01-19T04:56:28.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fc55d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08f415d540 0x7f08f40d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fc55d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f40cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fc55d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f40ce420 con 0x7f08f406baa0 2024-01-19T04:56:28.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fc55d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f415d920 con 0x7f08f415d540 2024-01-19T04:56:28.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fc55d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f406c040 con 0x7f08f40cda90 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fa2f9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08f415d540 0x7f08f40d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fa2f9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08f415d540 0x7f08f40d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58236/0 (socket says 172.21.15.31:58236) 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08fa2f9640 1 -- 172.21.15.31:0/3315265125 learned_addr learned my addr 172.21.15.31:0/3315265125 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.407+0000 7f08faafa640 1 --2- 172.21.15.31:0/3315265125 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f40cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.408+0000 7f08fa2f9640 1 -- 172.21.15.31:0/3315265125 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 msgr2=0x7f08f40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.408+0000 7f08fa2f9640 1 --2- 172.21.15.31:0/3315265125 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f40cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.408+0000 7f08fa2f9640 1 -- 172.21.15.31:0/3315265125 >> v1:172.21.15.137:6789/0 conn(0x7f08f406baa0 legacy=0x7f08f40cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.408+0000 7f08fa2f9640 1 -- 172.21.15.31:0/3315265125 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f406c320 con 0x7f08f415d540 2024-01-19T04:56:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.408+0000 7f08fa2f9640 1 --2- 172.21.15.31:0/3315265125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f40d0a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009870 tx=0x7f08ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fec96c34b8a0169 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.408+0000 7f08f92f7640 1 -- 172.21.15.31:0/3315265125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec031640 con 0x7f08f415d540 2024-01-19T04:56:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.408+0000 7f08f92f7640 1 -- 172.21.15.31:0/3315265125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0317a0 con 0x7f08f415d540 2024-01-19T04:56:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08f92f7640 1 -- 172.21.15.31:0/3315265125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec031b30 con 0x7f08f415d540 2024-01-19T04:56:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 -- 172.21.15.31:0/3315265125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 msgr2=0x7f08f40d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3315265125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f40d0a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009870 tx=0x7f08ec02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 -- 172.21.15.31:0/3315265125 shutdown_connections 2024-01-19T04:56:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3315265125 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f40cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3315265125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f40d0a70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 -- 172.21.15.31:0/3315265125 >> 172.21.15.31:0/3315265125 conn(0x7f08f40b2c60 msgr2=0x7f08f40b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 -- 172.21.15.31:0/3315265125 shutdown_connections 2024-01-19T04:56:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.409+0000 7f08fc55d640 1 -- 172.21.15.31:0/3315265125 wait complete. 2024-01-19T04:56:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.410+0000 7f08fc55d640 1 Processor -- start 2024-01-19T04:56:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.410+0000 7f08fc55d640 1 -- start start 2024-01-19T04:56:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08fc55d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f406baa0 0x7f08f415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08fc55d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f41f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08fc55d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f41f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08fc55d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415a7b0 con 0x7f08f415d540 2024-01-19T04:56:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08fc55d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415fc90 con 0x7f08f406baa0 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08fc55d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415f6d0 con 0x7f08f40cda90 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f41f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f41f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58250/0 (socket says 172.21.15.31:58250) 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 -- 172.21.15.31:0/3539793623 learned_addr learned my addr 172.21.15.31:0/3539793623 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08fa2f9640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f406baa0 0x7f08f415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08f9af8640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f41f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 -- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 msgr2=0x7f08f41f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f41f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 -- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f406baa0 msgr2=0x7f08f415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f406baa0 0x7f08f415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.411+0000 7f08faafa640 1 -- 172.21.15.31:0/3539793623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0094d0 con 0x7f08f415d540 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.412+0000 7f08faafa640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f41f2f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f08e4007e60 tx=0x7f08e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.412+0000 7f08f2ffd640 1 -- 172.21.15.31:0/3539793623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4014420 con 0x7f08f415d540 2024-01-19T04:56:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.412+0000 7f08f2ffd640 1 -- 172.21.15.31:0/3539793623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e40145a0 con 0x7f08f415d540 2024-01-19T04:56:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.412+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f415f9b0 con 0x7f08f415d540 2024-01-19T04:56:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.412+0000 7f08f2ffd640 1 -- 172.21.15.31:0/3539793623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4014950 con 0x7f08f415d540 2024-01-19T04:56:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.412+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f415f3f0 con 0x7f08f415d540 2024-01-19T04:56:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.413+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f4074a80 con 0x7f08f415d540 2024-01-19T04:56:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.413+0000 7f08f2ffd640 1 -- 172.21.15.31:0/3539793623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f08e4014ab0 con 0x7f08f415d540 2024-01-19T04:56:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.413+0000 7f08f2ffd640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f08e006aed0 0x7f08e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.414+0000 7f08f2ffd640 1 -- 172.21.15.31:0/3539793623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08e4089fc0 con 0x7f08f415d540 2024-01-19T04:56:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.414+0000 7f08fa2f9640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f08e006aed0 0x7f08e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.414+0000 7f08fa2f9640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f08e006aed0 0x7f08e006d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f08ec036000 tx=0x7f08ec03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.418+0000 7f08f2ffd640 1 -- 172.21.15.31:0/3539793623 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f08e4054e80 con 0x7f08f415d540 2024-01-19T04:56:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.599+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08f415a8f0 con 0x7f08f415d540 2024-01-19T04:56:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.600+0000 7f08f2ffd640 1 -- 172.21.15.31:0/3539793623 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f08f415a8f0 con 0x7f08f415d540 2024-01-19T04:56:28.600 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:28.600 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:26.929573+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f08e006aed0 msgr2=0x7f08e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f08e006aed0 0x7f08e006d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f08ec036000 tx=0x7f08ec03c040 comp rx=0 tx=0).stop 2024-01-19T04:56:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 msgr2=0x7f08f41f2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f41f2f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f08e4007e60 tx=0x7f08e400e4b0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 shutdown_connections 2024-01-19T04:56:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f08e006aed0 0x7f08e006d390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f415d540 0x7f08f41f2f90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08f40cda90 0x7f08f41f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 --2- 172.21.15.31:0/3539793623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f406baa0 0x7f08f415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 >> 172.21.15.31:0/3539793623 conn(0x7f08f40b2c60 msgr2=0x7f08f40d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 shutdown_connections 2024-01-19T04:56:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.602+0000 7f08fc55d640 1 -- 172.21.15.31:0/3539793623 wait complete. 2024-01-19T04:56:28.612 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-19T04:56:28.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:56:28.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.714+0000 7f2065e0f640 1 Processor -- start 2024-01-19T04:56:28.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.714+0000 7f2065e0f640 1 -- start start 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.714+0000 7f2065e0f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 0x7f2060160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.714+0000 7f2065e0f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600b3510 con 0x7f2060071df0 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.714+0000 7f2065e0f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600b37f0 con 0x7f206015d300 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.714+0000 7f2065e0f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20600b3ad0 con 0x7f20600b3030 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2060071df0 0x7f20600aeea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39626/0 (socket says 172.21.15.31:39626) 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205ffff640 1 -- 172.21.15.31:0/424860716 learned_addr learned my addr 172.21.15.31:0/424860716 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205effd640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 0x7f2060160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995656413 0 0) 0x7f20600b3510 con 0x7f2060071df0 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2048003520 con 0x7f2060071df0 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953343870 0 0) 0x7f20600b37f0 con 0x7f206015d300 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20600b3510 con 0x7f206015d300 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1379286563 0 0) 0x7f2048003520 con 0x7f2060071df0 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 msgr2=0x7f2060160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 0x7f2060160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 >> v1:172.21.15.137:6789/0 conn(0x7f206015d300 legacy=0x7f206006ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206006a650 con 0x7f2060071df0 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f20580019f0 con 0x7f2060071df0 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205effd640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 0x7f2060160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 >> v1:172.21.15.31:6789/0 conn(0x7f2060071df0 legacy=0x7f20600aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2048003b30 0x7f2048003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20480047c0 0x7f2048004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 0x7f2048005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600b3ad0 con 0x7f20480047c0 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204800b9b0 con 0x7f2048003b30 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204800bb70 con 0x7f20600b3030 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205effd640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20480047c0 0x7f2048004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205ffff640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 0x7f2048005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.715+0000 7f205f7fe640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2048003b30 0x7f2048003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.716+0000 7f205effd640 1 -- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 msgr2=0x7f2048005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.716+0000 7f205effd640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20600b3030 0x7f2048005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.716+0000 7f205effd640 1 -- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2048003b30 msgr2=0x7f2048003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.716+0000 7f205effd640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2048003b30 0x7f2048003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.716+0000 7f205effd640 1 -- 172.21.15.31:0/424860716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050009400 con 0x7f20480047c0 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.716+0000 7f205f7fe640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2048003b30 0x7f2048003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.716+0000 7f205effd640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20480047c0 0x7f2048004c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2050009c60 tx=0x7f20480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904226dc8d0470c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205002c8a0 con 0x7f20480047c0 2024-01-19T04:56:28.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205002ca00 con 0x7f20480047c0 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f205e7fc640 1 -- 172.21.15.31:0/424860716 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205002cdb0 con 0x7f20480047c0 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f2065e0f640 1 -- 172.21.15.31:0/424860716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20480047c0 msgr2=0x7f2048004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f2065e0f640 1 --2- 172.21.15.31:0/424860716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20480047c0 0x7f2048004c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2050009c60 tx=0x7f20480057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f205ffff640 1 -- 172.21.15.31:0/424860716 reap_dead start 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f2065e0f640 1 -- 172.21.15.31:0/424860716 shutdown_connections 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f2065e0f640 1 -- 172.21.15.31:0/424860716 >> 172.21.15.31:0/424860716 conn(0x7f206006d750 msgr2=0x7f206006db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f2065e0f640 1 -- 172.21.15.31:0/424860716 shutdown_connections 2024-01-19T04:56:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.717+0000 7f2065e0f640 1 -- 172.21.15.31:0/424860716 wait complete. 2024-01-19T04:56:28.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.718+0000 7f2065e0f640 1 Processor -- start 2024-01-19T04:56:28.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.718+0000 7f2065e0f640 1 -- start start 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f2065e0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20480047c0 0x7f2060149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f2065e0f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2060159a60 0x7f206015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f2065e0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206015b9a0 0x7f206015be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f2065e0f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c4fa0 con 0x7f206015b9a0 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f2065e0f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060159ee0 con 0x7f20480047c0 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f2065e0f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206015a160 con 0x7f2060159a60 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f205ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206015b9a0 0x7f206015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f205ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206015b9a0 0x7f206015be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58272/0 (socket says 172.21.15.31:58272) 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.719+0000 7f205ffff640 1 -- 172.21.15.31:0/3581059081 learned_addr learned my addr 172.21.15.31:0/3581059081 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f205ffff640 1 -- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2060159a60 msgr2=0x7f206015b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f205ffff640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2060159a60 0x7f206015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f205ffff640 1 -- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20480047c0 msgr2=0x7f2060149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f205ffff640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20480047c0 0x7f2060149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f205ffff640 1 -- 172.21.15.31:0/3581059081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206015a440 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f205ffff640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206015b9a0 0x7f206015be20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2058000f80 tx=0x7f2058007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f2047fff640 1 -- 172.21.15.31:0/3581059081 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800ab90 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f2047fff640 1 -- 172.21.15.31:0/3581059081 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205800ad50 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206015a720 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f2047fff640 1 -- 172.21.15.31:0/3581059081 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058013630 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.720+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20601f6710 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.721+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20600696e0 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.721+0000 7f2047fff640 1 -- 172.21.15.31:0/3581059081 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2058013790 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.722+0000 7f2047fff640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f204c06aed0 0x7f204c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.722+0000 7f2047fff640 1 -- 172.21.15.31:0/3581059081 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2058089d60 con 0x7f206015b9a0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.722+0000 7f205f7fe640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f204c06aed0 0x7f204c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.722+0000 7f205f7fe640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f204c06aed0 0x7f204c06d390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2054007fd0 tx=0x7f2054009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:28.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.726+0000 7f2047fff640 1 -- 172.21.15.31:0/3581059081 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2058054b90 con 0x7f206015b9a0 2024-01-19T04:56:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.901+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f206006b280 con 0x7f204c06aed0 2024-01-19T04:56:28.906 INFO:tasks.ceph.mds.a.smithi031.stderr:2024-01-19T04:56:28.906+0000 7f6a83297640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-19T04:56:28.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.909+0000 7f2047fff640 1 -- 172.21.15.31:0/3581059081 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289113 (secure 0 0 0) 0x7f2054001c80 con 0x7f204c06aed0 2024-01-19T04:56:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:28.914 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:56:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f204c06aed0 msgr2=0x7f204c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f204c06aed0 0x7f204c06d390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2054007fd0 tx=0x7f2054009040 comp rx=0 tx=0).stop 2024-01-19T04:56:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206015b9a0 msgr2=0x7f206015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206015b9a0 0x7f206015be20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2058000f80 tx=0x7f2058007bd0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 shutdown_connections 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f204c06aed0 0x7f204c06d390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206015b9a0 0x7f206015be20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2060159a60 0x7f206015b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 --2- 172.21.15.31:0/3581059081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20480047c0 0x7f2060149b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 >> 172.21.15.31:0/3581059081 conn(0x7f206006d750 msgr2=0x7f206006e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 shutdown_connections 2024-01-19T04:56:28.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:28.912+0000 7f2065e0f640 1 -- 172.21.15.31:0/3581059081 wait complete. 2024-01-19T04:56:28.922 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-01-19T04:56:27.576917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220284,"kb_used_data":4108,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754436,"statfs":{"total":773094113280,"available":772868542464,"internally_reserved":0,"allocated":4206592,"data_stored":2342003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000917"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943722+0000","last_change":"2024-01-19T04:56:25.938864+0000","last_active":"2024-01-19T04:56:25.943722+0000","last_peered":"2024-01-19T04:56:25.943722+0000","last_clean":"2024-01-19T04:56:25.943722+0000","last_became_active":"2024-01-19T04:56:25.938572+0000","last_became_peered":"2024-01-19T04:56:25.938572+0000","last_unstale":"2024-01-19T04:56:25.943722+0000","last_undegraded":"2024-01-19T04:56:25.943722+0000","last_fullsized":"2024-01-19T04:56:25.943722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:14.516624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944322+0000","last_change":"2024-01-19T04:56:25.941620+0000","last_active":"2024-01-19T04:56:25.944322+0000","last_peered":"2024-01-19T04:56:25.944322+0000","last_clean":"2024-01-19T04:56:25.944322+0000","last_became_active":"2024-01-19T04:56:25.941100+0000","last_became_peered":"2024-01-19T04:56:25.941100+0000","last_unstale":"2024-01-19T04:56:25.944322+0000","last_undegraded":"2024-01-19T04:56:25.944322+0000","last_fullsized":"2024-01-19T04:56:25.944322+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:13:42.229617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950804+0000","last_change":"2024-01-19T04:56:25.945484+0000","last_active":"2024-01-19T04:56:25.950804+0000","last_peered":"2024-01-19T04:56:25.950804+0000","last_clean":"2024-01-19T04:56:25.950804+0000","last_became_active":"2024-01-19T04:56:25.944935+0000","last_became_peered":"2024-01-19T04:56:25.944935+0000","last_unstale":"2024-01-19T04:56:25.950804+0000","last_undegraded":"2024-01-19T04:56:25.950804+0000","last_fullsized":"2024-01-19T04:56:25.950804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:58:19.400385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943888+0000","last_change":"2024-01-19T04:56:25.941103+0000","last_active":"2024-01-19T04:56:25.943888+0000","last_peered":"2024-01-19T04:56:25.943888+0000","last_clean":"2024-01-19T04:56:25.943888+0000","last_became_active":"2024-01-19T04:56:25.940387+0000","last_became_peered":"2024-01-19T04:56:25.940387+0000","last_unstale":"2024-01-19T04:56:25.943888+0000","last_undegraded":"2024-01-19T04:56:25.943888+0000","last_fullsized":"2024-01-19T04:56:25.943888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946892+0000","last_change":"2024-01-19T04:56:25.940429+0000","last_active":"2024-01-19T04:56:25.946892+0000","last_peered":"2024-01-19T04:56:25.946892+0000","last_clean":"2024-01-19T04:56:25.946892+0000","last_became_active":"2024-01-19T04:56:25.940179+0000","last_became_peered":"2024-01-19T04:56:25.940179+0000","last_unstale":"2024-01-19T04:56:25.946892+0000","last_undegraded":"2024-01-19T04:56:25.946892+0000","last_fullsized":"2024-01-19T04:56:25.946892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:00:17.149689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943679+0000","last_change":"2024-01-19T04:56:25.935733+0000","last_active":"2024-01-19T04:56:25.943679+0000","last_peered":"2024-01-19T04:56:25.943679+0000","last_clean":"2024-01-19T04:56:25.943679+0000","last_became_active":"2024-01-19T04:56:25.935422+0000","last_became_peered":"2024-01-19T04:56:25.935422+0000","last_unstale":"2024-01-19T04:56:25.943679+0000","last_undegraded":"2024-01-19T04:56:25.943679+0000","last_fullsized":"2024-01-19T04:56:25.943679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:12:35.815320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950700+0000","last_change":"2024-01-19T04:56:25.945826+0000","last_active":"2024-01-19T04:56:25.950700+0000","last_peered":"2024-01-19T04:56:25.950700+0000","last_clean":"2024-01-19T04:56:25.950700+0000","last_became_active":"2024-01-19T04:56:25.945218+0000","last_became_peered":"2024-01-19T04:56:25.945218+0000","last_unstale":"2024-01-19T04:56:25.950700+0000","last_undegraded":"2024-01-19T04:56:25.950700+0000","last_fullsized":"2024-01-19T04:56:25.950700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:24.207122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944932+0000","last_change":"2024-01-19T04:56:25.942901+0000","last_active":"2024-01-19T04:56:25.944932+0000","last_peered":"2024-01-19T04:56:25.944932+0000","last_clean":"2024-01-19T04:56:25.944932+0000","last_became_active":"2024-01-19T04:56:25.942291+0000","last_became_peered":"2024-01-19T04:56:25.942291+0000","last_unstale":"2024-01-19T04:56:25.944932+0000","last_undegraded":"2024-01-19T04:56:25.944932+0000","last_fullsized":"2024-01-19T04:56:25.944932+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950647+0000","last_change":"2024-01-19T04:56:25.944837+0000","last_active":"2024-01-19T04:56:25.950647+0000","last_peered":"2024-01-19T04:56:25.950647+0000","last_clean":"2024-01-19T04:56:25.950647+0000","last_became_active":"2024-01-19T04:56:25.944549+0000","last_became_peered":"2024-01-19T04:56:25.944549+0000","last_unstale":"2024-01-19T04:56:25.950647+0000","last_undegraded":"2024-01-19T04:56:25.950647+0000","last_fullsized":"2024-01-19T04:56:25.950647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:11.285759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953835+0000","last_change":"2024-01-19T04:56:25.950327+0000","last_active":"2024-01-19T04:56:25.953835+0000","last_peered":"2024-01-19T04:56:25.953835+0000","last_clean":"2024-01-19T04:56:25.953835+0000","last_became_active":"2024-01-19T04:56:25.950027+0000","last_became_peered":"2024-01-19T04:56:25.950027+0000","last_unstale":"2024-01-19T04:56:25.953835+0000","last_undegraded":"2024-01-19T04:56:25.953835+0000","last_fullsized":"2024-01-19T04:56:25.953835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:49.034540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950610+0000","last_change":"2024-01-19T04:56:25.945557+0000","last_active":"2024-01-19T04:56:25.950610+0000","last_peered":"2024-01-19T04:56:25.950610+0000","last_clean":"2024-01-19T04:56:25.950610+0000","last_became_active":"2024-01-19T04:56:25.945267+0000","last_became_peered":"2024-01-19T04:56:25.945267+0000","last_unstale":"2024-01-19T04:56:25.950610+0000","last_undegraded":"2024-01-19T04:56:25.950610+0000","last_fullsized":"2024-01-19T04:56:25.950610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:49.034540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950894+0000","last_change":"2024-01-19T04:56:25.946677+0000","last_active":"2024-01-19T04:56:25.950894+0000","last_peered":"2024-01-19T04:56:25.950894+0000","last_clean":"2024-01-19T04:56:25.950894+0000","last_became_active":"2024-01-19T04:56:25.946372+0000","last_became_peered":"2024-01-19T04:56:25.946372+0000","last_unstale":"2024-01-19T04:56:25.950894+0000","last_undegraded":"2024-01-19T04:56:25.950894+0000","last_fullsized":"2024-01-19T04:56:25.950894+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:30:16.180514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947391+0000","last_change":"2024-01-19T04:56:25.945260+0000","last_active":"2024-01-19T04:56:25.947391+0000","last_peered":"2024-01-19T04:56:25.947391+0000","last_clean":"2024-01-19T04:56:25.947391+0000","last_became_active":"2024-01-19T04:56:25.945002+0000","last_became_peered":"2024-01-19T04:56:25.945002+0000","last_unstale":"2024-01-19T04:56:25.947391+0000","last_undegraded":"2024-01-19T04:56:25.947391+0000","last_fullsized":"2024-01-19T04:56:25.947391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:57:31.321604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944622+0000","last_change":"2024-01-19T04:56:25.939505+0000","last_active":"2024-01-19T04:56:25.944622+0000","last_peered":"2024-01-19T04:56:25.944622+0000","last_clean":"2024-01-19T04:56:25.944622+0000","last_became_active":"2024-01-19T04:56:25.938996+0000","last_became_peered":"2024-01-19T04:56:25.938996+0000","last_unstale":"2024-01-19T04:56:25.944622+0000","last_undegraded":"2024-01-19T04:56:25.944622+0000","last_fullsized":"2024-01-19T04:56:25.944622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:30:16.180514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945007+0000","last_change":"2024-01-19T04:56:25.940633+0000","last_active":"2024-01-19T04:56:25.945007+0000","last_peered":"2024-01-19T04:56:25.945007+0000","last_clean":"2024-01-19T04:56:25.945007+0000","last_became_active":"2024-01-19T04:56:25.940373+0000","last_became_peered":"2024-01-19T04:56:25.940373+0000","last_unstale":"2024-01-19T04:56:25.945007+0000","last_undegraded":"2024-01-19T04:56:25.945007+0000","last_fullsized":"2024-01-19T04:56:25.945007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:49.034540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953289+0000","last_change":"2024-01-19T04:56:25.950156+0000","last_active":"2024-01-19T04:56:25.953289+0000","last_peered":"2024-01-19T04:56:25.953289+0000","last_clean":"2024-01-19T04:56:25.953289+0000","last_became_active":"2024-01-19T04:56:25.949814+0000","last_became_peered":"2024-01-19T04:56:25.949814+0000","last_unstale":"2024-01-19T04:56:25.953289+0000","last_undegraded":"2024-01-19T04:56:25.953289+0000","last_fullsized":"2024-01-19T04:56:25.953289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:12:35.815320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946372+0000","last_change":"2024-01-19T04:56:25.944476+0000","last_active":"2024-01-19T04:56:25.946372+0000","last_peered":"2024-01-19T04:56:25.946372+0000","last_clean":"2024-01-19T04:56:25.946372+0000","last_became_active":"2024-01-19T04:56:25.943852+0000","last_became_peered":"2024-01-19T04:56:25.943852+0000","last_unstale":"2024-01-19T04:56:25.946372+0000","last_undegraded":"2024-01-19T04:56:25.946372+0000","last_fullsized":"2024-01-19T04:56:25.946372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:27.173131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954125+0000","last_change":"2024-01-19T04:56:25.948730+0000","last_active":"2024-01-19T04:56:25.954125+0000","last_peered":"2024-01-19T04:56:25.954125+0000","last_clean":"2024-01-19T04:56:25.954125+0000","last_became_active":"2024-01-19T04:56:25.948458+0000","last_became_peered":"2024-01-19T04:56:25.948458+0000","last_unstale":"2024-01-19T04:56:25.954125+0000","last_undegraded":"2024-01-19T04:56:25.954125+0000","last_fullsized":"2024-01-19T04:56:25.954125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:22:02.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.948489+0000","last_change":"2024-01-19T04:56:25.946921+0000","last_active":"2024-01-19T04:56:25.948489+0000","last_peered":"2024-01-19T04:56:25.948489+0000","last_clean":"2024-01-19T04:56:25.948489+0000","last_became_active":"2024-01-19T04:56:25.946662+0000","last_became_peered":"2024-01-19T04:56:25.946662+0000","last_unstale":"2024-01-19T04:56:25.948489+0000","last_undegraded":"2024-01-19T04:56:25.948489+0000","last_fullsized":"2024-01-19T04:56:25.948489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:06:15.316666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947044+0000","last_change":"2024-01-19T04:56:25.942039+0000","last_active":"2024-01-19T04:56:25.947044+0000","last_peered":"2024-01-19T04:56:25.947044+0000","last_clean":"2024-01-19T04:56:25.947044+0000","last_became_active":"2024-01-19T04:56:25.941715+0000","last_became_peered":"2024-01-19T04:56:25.941715+0000","last_unstale":"2024-01-19T04:56:25.947044+0000","last_undegraded":"2024-01-19T04:56:25.947044+0000","last_fullsized":"2024-01-19T04:56:25.947044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:28:21.687396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.939397+0000","last_change":"2024-01-19T04:56:25.934401+0000","last_active":"2024-01-19T04:56:25.939397+0000","last_peered":"2024-01-19T04:56:25.939397+0000","last_clean":"2024-01-19T04:56:25.939397+0000","last_became_active":"2024-01-19T04:56:25.934134+0000","last_became_peered":"2024-01-19T04:56:25.934134+0000","last_unstale":"2024-01-19T04:56:25.939397+0000","last_undegraded":"2024-01-19T04:56:25.939397+0000","last_fullsized":"2024-01-19T04:56:25.939397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:13:57.205481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944421+0000","last_change":"2024-01-19T04:56:25.942496+0000","last_active":"2024-01-19T04:56:25.944421+0000","last_peered":"2024-01-19T04:56:25.944421+0000","last_clean":"2024-01-19T04:56:25.944421+0000","last_became_active":"2024-01-19T04:56:25.942202+0000","last_became_peered":"2024-01-19T04:56:25.942202+0000","last_unstale":"2024-01-19T04:56:25.944421+0000","last_undegraded":"2024-01-19T04:56:25.944421+0000","last_fullsized":"2024-01-19T04:56:25.944421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:02.088796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.933743+0000","last_change":"2024-01-19T04:56:25.932844+0000","last_active":"2024-01-19T04:56:25.933743+0000","last_peered":"2024-01-19T04:56:25.933743+0000","last_clean":"2024-01-19T04:56:25.933743+0000","last_became_active":"2024-01-19T04:56:25.932606+0000","last_became_peered":"2024-01-19T04:56:25.932606+0000","last_unstale":"2024-01-19T04:56:25.933743+0000","last_undegraded":"2024-01-19T04:56:25.933743+0000","last_fullsized":"2024-01-19T04:56:25.933743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:17.691346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.942752+0000","last_change":"2024-01-19T04:56:25.938490+0000","last_active":"2024-01-19T04:56:25.942752+0000","last_peered":"2024-01-19T04:56:25.942752+0000","last_clean":"2024-01-19T04:56:25.942752+0000","last_became_active":"2024-01-19T04:56:25.938151+0000","last_became_peered":"2024-01-19T04:56:25.938151+0000","last_unstale":"2024-01-19T04:56:25.942752+0000","last_undegraded":"2024-01-19T04:56:25.942752+0000","last_fullsized":"2024-01-19T04:56:25.942752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:27.173131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945652+0000","last_change":"2024-01-19T04:56:25.943682+0000","last_active":"2024-01-19T04:56:25.945652+0000","last_peered":"2024-01-19T04:56:25.945652+0000","last_clean":"2024-01-19T04:56:25.945652+0000","last_became_active":"2024-01-19T04:56:25.943356+0000","last_became_peered":"2024-01-19T04:56:25.943356+0000","last_unstale":"2024-01-19T04:56:25.945652+0000","last_undegraded":"2024-01-19T04:56:25.945652+0000","last_fullsized":"2024-01-19T04:56:25.945652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:27.173131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944300+0000","last_change":"2024-01-19T04:56:25.941402+0000","last_active":"2024-01-19T04:56:25.944300+0000","last_peered":"2024-01-19T04:56:25.944300+0000","last_clean":"2024-01-19T04:56:25.944300+0000","last_became_active":"2024-01-19T04:56:25.940766+0000","last_became_peered":"2024-01-19T04:56:25.940766+0000","last_unstale":"2024-01-19T04:56:25.944300+0000","last_undegraded":"2024-01-19T04:56:25.944300+0000","last_fullsized":"2024-01-19T04:56:25.944300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:43:57.779892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944180+0000","last_change":"2024-01-19T04:56:25.941547+0000","last_active":"2024-01-19T04:56:25.944180+0000","last_peered":"2024-01-19T04:56:25.944180+0000","last_clean":"2024-01-19T04:56:25.944180+0000","last_became_active":"2024-01-19T04:56:25.940049+0000","last_became_peered":"2024-01-19T04:56:25.940049+0000","last_unstale":"2024-01-19T04:56:25.944180+0000","last_undegraded":"2024-01-19T04:56:25.944180+0000","last_fullsized":"2024-01-19T04:56:25.944180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:02.466345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.931653+0000","last_change":"2024-01-19T04:56:25.929826+0000","last_active":"2024-01-19T04:56:25.931653+0000","last_peered":"2024-01-19T04:56:25.931653+0000","last_clean":"2024-01-19T04:56:25.931653+0000","last_became_active":"2024-01-19T04:56:25.929467+0000","last_became_peered":"2024-01-19T04:56:25.929467+0000","last_unstale":"2024-01-19T04:56:25.931653+0000","last_undegraded":"2024-01-19T04:56:25.931653+0000","last_fullsized":"2024-01-19T04:56:25.931653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944464+0000","last_change":"2024-01-19T04:56:25.942031+0000","last_active":"2024-01-19T04:56:25.944464+0000","last_peered":"2024-01-19T04:56:25.944464+0000","last_clean":"2024-01-19T04:56:25.944464+0000","last_became_active":"2024-01-19T04:56:25.941785+0000","last_became_peered":"2024-01-19T04:56:25.941785+0000","last_unstale":"2024-01-19T04:56:25.944464+0000","last_undegraded":"2024-01-19T04:56:25.944464+0000","last_fullsized":"2024-01-19T04:56:25.944464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:14.516624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945800+0000","last_change":"2024-01-19T04:56:25.943449+0000","last_active":"2024-01-19T04:56:25.945800+0000","last_peered":"2024-01-19T04:56:25.945800+0000","last_clean":"2024-01-19T04:56:25.945800+0000","last_became_active":"2024-01-19T04:56:25.943195+0000","last_became_peered":"2024-01-19T04:56:25.943195+0000","last_unstale":"2024-01-19T04:56:25.945800+0000","last_undegraded":"2024-01-19T04:56:25.945800+0000","last_fullsized":"2024-01-19T04:56:25.945800+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:38.155889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950749+0000","last_change":"2024-01-19T04:56:25.948337+0000","last_active":"2024-01-19T04:56:25.950749+0000","last_peered":"2024-01-19T04:56:25.950749+0000","last_clean":"2024-01-19T04:56:25.950749+0000","last_became_active":"2024-01-19T04:56:25.947932+0000","last_became_peered":"2024-01-19T04:56:25.947932+0000","last_unstale":"2024-01-19T04:56:25.950749+0000","last_undegraded":"2024-01-19T04:56:25.950749+0000","last_fullsized":"2024-01-19T04:56:25.950749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:43:01.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943425+0000","last_change":"2024-01-19T04:56:25.941922+0000","last_active":"2024-01-19T04:56:25.943425+0000","last_peered":"2024-01-19T04:56:25.943425+0000","last_clean":"2024-01-19T04:56:25.943425+0000","last_became_active":"2024-01-19T04:56:25.941636+0000","last_became_peered":"2024-01-19T04:56:25.941636+0000","last_unstale":"2024-01-19T04:56:25.943425+0000","last_undegraded":"2024-01-19T04:56:25.943425+0000","last_fullsized":"2024-01-19T04:56:25.943425+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:04.091802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.933743+0000","last_change":"2024-01-19T04:56:25.932234+0000","last_active":"2024-01-19T04:56:25.933743+0000","last_peered":"2024-01-19T04:56:25.933743+0000","last_clean":"2024-01-19T04:56:25.933743+0000","last_became_active":"2024-01-19T04:56:25.931978+0000","last_became_peered":"2024-01-19T04:56:25.931978+0000","last_unstale":"2024-01-19T04:56:25.933743+0000","last_undegraded":"2024-01-19T04:56:25.933743+0000","last_fullsized":"2024-01-19T04:56:25.933743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:59:03.271823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953477+0000","last_change":"2024-01-19T04:56:24.926873+0000","last_active":"2024-01-19T04:56:25.953477+0000","last_peered":"2024-01-19T04:56:25.953477+0000","last_clean":"2024-01-19T04:56:25.953477+0000","last_became_active":"2024-01-19T04:56:24.926598+0000","last_became_peered":"2024-01-19T04:56:24.926598+0000","last_unstale":"2024-01-19T04:56:25.953477+0000","last_undegraded":"2024-01-19T04:56:25.953477+0000","last_fullsized":"2024-01-19T04:56:25.953477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:33:46.879094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944089+0000","last_change":"2024-01-19T04:56:25.942560+0000","last_active":"2024-01-19T04:56:25.944089+0000","last_peered":"2024-01-19T04:56:25.944089+0000","last_clean":"2024-01-19T04:56:25.944089+0000","last_became_active":"2024-01-19T04:56:25.942313+0000","last_became_peered":"2024-01-19T04:56:25.942313+0000","last_unstale":"2024-01-19T04:56:25.944089+0000","last_undegraded":"2024-01-19T04:56:25.944089+0000","last_fullsized":"2024-01-19T04:56:25.944089+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:11.285759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.949143+0000","last_change":"2024-01-19T04:56:24.930063+0000","last_active":"2024-01-19T04:56:25.949143+0000","last_peered":"2024-01-19T04:56:25.949143+0000","last_clean":"2024-01-19T04:56:25.949143+0000","last_became_active":"2024-01-19T04:56:24.929785+0000","last_became_peered":"2024-01-19T04:56:24.929785+0000","last_unstale":"2024-01-19T04:56:25.949143+0000","last_undegraded":"2024-01-19T04:56:25.949143+0000","last_fullsized":"2024-01-19T04:56:25.949143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:10.280240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953260+0000","last_change":"2024-01-19T04:56:25.950148+0000","last_active":"2024-01-19T04:56:25.953260+0000","last_peered":"2024-01-19T04:56:25.953260+0000","last_clean":"2024-01-19T04:56:25.953260+0000","last_became_active":"2024-01-19T04:56:25.949760+0000","last_became_peered":"2024-01-19T04:56:25.949760+0000","last_unstale":"2024-01-19T04:56:25.953260+0000","last_undegraded":"2024-01-19T04:56:25.953260+0000","last_fullsized":"2024-01-19T04:56:25.953260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:33.422822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954694+0000","last_change":"2024-01-19T04:56:24.927145+0000","last_active":"2024-01-19T04:56:25.954694+0000","last_peered":"2024-01-19T04:56:25.954694+0000","last_clean":"2024-01-19T04:56:25.954694+0000","last_became_active":"2024-01-19T04:56:24.926862+0000","last_became_peered":"2024-01-19T04:56:24.926862+0000","last_unstale":"2024-01-19T04:56:25.954694+0000","last_undegraded":"2024-01-19T04:56:25.954694+0000","last_fullsized":"2024-01-19T04:56:25.954694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:03.086283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944129+0000","last_change":"2024-01-19T04:56:25.938848+0000","last_active":"2024-01-19T04:56:25.944129+0000","last_peered":"2024-01-19T04:56:25.944129+0000","last_clean":"2024-01-19T04:56:25.944129+0000","last_became_active":"2024-01-19T04:56:25.938508+0000","last_became_peered":"2024-01-19T04:56:25.938508+0000","last_unstale":"2024-01-19T04:56:25.944129+0000","last_undegraded":"2024-01-19T04:56:25.944129+0000","last_fullsized":"2024-01-19T04:56:25.944129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:17.691346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947135+0000","last_change":"2024-01-19T04:56:24.931702+0000","last_active":"2024-01-19T04:56:25.947135+0000","last_peered":"2024-01-19T04:56:25.947135+0000","last_clean":"2024-01-19T04:56:25.947135+0000","last_became_active":"2024-01-19T04:56:24.931369+0000","last_became_peered":"2024-01-19T04:56:24.931369+0000","last_unstale":"2024-01-19T04:56:25.947135+0000","last_undegraded":"2024-01-19T04:56:25.947135+0000","last_fullsized":"2024-01-19T04:56:25.947135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:30:56.438301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.941883+0000","last_change":"2024-01-19T04:56:25.940964+0000","last_active":"2024-01-19T04:56:25.941883+0000","last_peered":"2024-01-19T04:56:25.941883+0000","last_clean":"2024-01-19T04:56:25.941883+0000","last_became_active":"2024-01-19T04:56:25.940589+0000","last_became_peered":"2024-01-19T04:56:25.940589+0000","last_unstale":"2024-01-19T04:56:25.941883+0000","last_undegraded":"2024-01-19T04:56:25.941883+0000","last_fullsized":"2024-01-19T04:56:25.941883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:17.691346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.951984+0000","last_change":"2024-01-19T04:56:24.925660+0000","last_active":"2024-01-19T04:56:25.951984+0000","last_peered":"2024-01-19T04:56:25.951984+0000","last_clean":"2024-01-19T04:56:25.951984+0000","last_became_active":"2024-01-19T04:56:24.925394+0000","last_became_peered":"2024-01-19T04:56:24.925394+0000","last_unstale":"2024-01-19T04:56:25.951984+0000","last_undegraded":"2024-01-19T04:56:25.951984+0000","last_fullsized":"2024-01-19T04:56:25.951984+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:48:32.505270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943198+0000","last_change":"2024-01-19T04:56:25.936910+0000","last_active":"2024-01-19T04:56:25.943198+0000","last_peered":"2024-01-19T04:56:25.943198+0000","last_clean":"2024-01-19T04:56:25.943198+0000","last_became_active":"2024-01-19T04:56:25.936658+0000","last_became_peered":"2024-01-19T04:56:25.936658+0000","last_unstale":"2024-01-19T04:56:25.943198+0000","last_undegraded":"2024-01-19T04:56:25.943198+0000","last_fullsized":"2024-01-19T04:56:25.943198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:38.155889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954265+0000","last_change":"2024-01-19T04:56:24.925809+0000","last_active":"2024-01-19T04:56:25.954265+0000","last_peered":"2024-01-19T04:56:25.954265+0000","last_clean":"2024-01-19T04:56:25.954265+0000","last_became_active":"2024-01-19T04:56:24.925436+0000","last_became_peered":"2024-01-19T04:56:24.925436+0000","last_unstale":"2024-01-19T04:56:25.954265+0000","last_undegraded":"2024-01-19T04:56:25.954265+0000","last_fullsized":"2024-01-19T04:56:25.954265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:04:44.398370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.941260+0000","last_change":"2024-01-19T04:56:25.939555+0000","last_active":"2024-01-19T04:56:25.941260+0000","last_peered":"2024-01-19T04:56:25.941260+0000","last_clean":"2024-01-19T04:56:25.941260+0000","last_became_active":"2024-01-19T04:56:25.939270+0000","last_became_peered":"2024-01-19T04:56:25.939270+0000","last_unstale":"2024-01-19T04:56:25.941260+0000","last_undegraded":"2024-01-19T04:56:25.941260+0000","last_fullsized":"2024-01-19T04:56:25.941260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:33:47.884613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926126+0000","last_change":"2024-01-19T04:56:24.923195+0000","last_active":"2024-01-19T04:56:25.926126+0000","last_peered":"2024-01-19T04:56:25.926126+0000","last_clean":"2024-01-19T04:56:25.926126+0000","last_became_active":"2024-01-19T04:56:24.922941+0000","last_became_peered":"2024-01-19T04:56:24.922941+0000","last_unstale":"2024-01-19T04:56:25.926126+0000","last_undegraded":"2024-01-19T04:56:25.926126+0000","last_fullsized":"2024-01-19T04:56:25.926126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:53.631079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943111+0000","last_change":"2024-01-19T04:56:25.934541+0000","last_active":"2024-01-19T04:56:25.943111+0000","last_peered":"2024-01-19T04:56:25.943111+0000","last_clean":"2024-01-19T04:56:25.943111+0000","last_became_active":"2024-01-19T04:56:25.934277+0000","last_became_peered":"2024-01-19T04:56:25.934277+0000","last_unstale":"2024-01-19T04:56:25.943111+0000","last_undegraded":"2024-01-19T04:56:25.943111+0000","last_fullsized":"2024-01-19T04:56:25.943111+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:59:03.271823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.925715+0000","last_change":"2024-01-19T04:56:24.924504+0000","last_active":"2024-01-19T04:56:25.925715+0000","last_peered":"2024-01-19T04:56:25.925715+0000","last_clean":"2024-01-19T04:56:25.925715+0000","last_became_active":"2024-01-19T04:56:24.924223+0000","last_became_peered":"2024-01-19T04:56:24.924223+0000","last_unstale":"2024-01-19T04:56:25.925715+0000","last_undegraded":"2024-01-19T04:56:25.925715+0000","last_fullsized":"2024-01-19T04:56:25.925715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:18:41.438563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950847+0000","last_change":"2024-01-19T04:56:25.945986+0000","last_active":"2024-01-19T04:56:25.950847+0000","last_peered":"2024-01-19T04:56:25.950847+0000","last_clean":"2024-01-19T04:56:25.950847+0000","last_became_active":"2024-01-19T04:56:25.945742+0000","last_became_peered":"2024-01-19T04:56:25.945742+0000","last_unstale":"2024-01-19T04:56:25.950847+0000","last_undegraded":"2024-01-19T04:56:25.950847+0000","last_fullsized":"2024-01-19T04:56:25.950847+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:12:35.815320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.927045+0000","last_change":"2024-01-19T04:56:24.928679+0000","last_active":"2024-01-19T04:56:25.927045+0000","last_peered":"2024-01-19T04:56:25.927045+0000","last_clean":"2024-01-19T04:56:25.927045+0000","last_became_active":"2024-01-19T04:56:24.928405+0000","last_became_peered":"2024-01-19T04:56:24.928405+0000","last_unstale":"2024-01-19T04:56:25.927045+0000","last_undegraded":"2024-01-19T04:56:25.927045+0000","last_fullsized":"2024-01-19T04:56:25.927045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:23.201603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954095+0000","last_change":"2024-01-19T04:56:25.948766+0000","last_active":"2024-01-19T04:56:25.954095+0000","last_peered":"2024-01-19T04:56:25.954095+0000","last_clean":"2024-01-19T04:56:25.954095+0000","last_became_active":"2024-01-19T04:56:25.948508+0000","last_became_peered":"2024-01-19T04:56:25.948508+0000","last_unstale":"2024-01-19T04:56:25.954095+0000","last_undegraded":"2024-01-19T04:56:25.954095+0000","last_fullsized":"2024-01-19T04:56:25.954095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:12:17.004437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.923542+0000","last_change":"2024-01-19T04:56:24.925942+0000","last_active":"2024-01-19T04:56:25.923542+0000","last_peered":"2024-01-19T04:56:25.923542+0000","last_clean":"2024-01-19T04:56:25.923542+0000","last_became_active":"2024-01-19T04:56:24.925656+0000","last_became_peered":"2024-01-19T04:56:24.925656+0000","last_unstale":"2024-01-19T04:56:25.923542+0000","last_undegraded":"2024-01-19T04:56:25.923542+0000","last_fullsized":"2024-01-19T04:56:25.923542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:32.417303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944856+0000","last_change":"2024-01-19T04:56:25.933886+0000","last_active":"2024-01-19T04:56:25.944856+0000","last_peered":"2024-01-19T04:56:25.944856+0000","last_clean":"2024-01-19T04:56:25.944856+0000","last_became_active":"2024-01-19T04:56:25.933586+0000","last_became_peered":"2024-01-19T04:56:25.933586+0000","last_unstale":"2024-01-19T04:56:25.944856+0000","last_undegraded":"2024-01-19T04:56:25.944856+0000","last_fullsized":"2024-01-19T04:56:25.944856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:58:19.400385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950999+0000","last_change":"2024-01-19T04:56:24.930228+0000","last_active":"2024-01-19T04:56:25.950999+0000","last_peered":"2024-01-19T04:56:25.950999+0000","last_clean":"2024-01-19T04:56:25.950999+0000","last_became_active":"2024-01-19T04:56:24.929966+0000","last_became_peered":"2024-01-19T04:56:24.929966+0000","last_unstale":"2024-01-19T04:56:25.950999+0000","last_undegraded":"2024-01-19T04:56:25.950999+0000","last_fullsized":"2024-01-19T04:56:25.950999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:03.086283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950825+0000","last_change":"2024-01-19T04:56:25.945682+0000","last_active":"2024-01-19T04:56:25.950825+0000","last_peered":"2024-01-19T04:56:25.950825+0000","last_clean":"2024-01-19T04:56:25.950825+0000","last_became_active":"2024-01-19T04:56:25.945308+0000","last_became_peered":"2024-01-19T04:56:25.945308+0000","last_unstale":"2024-01-19T04:56:25.950825+0000","last_undegraded":"2024-01-19T04:56:25.950825+0000","last_fullsized":"2024-01-19T04:56:25.950825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:43:41.276962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.959410+0000","last_change":"2024-01-19T04:56:24.932470+0000","last_active":"2024-01-19T04:56:25.959410+0000","last_peered":"2024-01-19T04:56:25.959410+0000","last_clean":"2024-01-19T04:56:25.959410+0000","last_became_active":"2024-01-19T04:56:24.930719+0000","last_became_peered":"2024-01-19T04:56:24.930719+0000","last_unstale":"2024-01-19T04:56:25.959410+0000","last_undegraded":"2024-01-19T04:56:25.959410+0000","last_fullsized":"2024-01-19T04:56:25.959410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:26.167612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945347+0000","last_change":"2024-01-19T04:56:25.943515+0000","last_active":"2024-01-19T04:56:25.945347+0000","last_peered":"2024-01-19T04:56:25.945347+0000","last_clean":"2024-01-19T04:56:25.945347+0000","last_became_active":"2024-01-19T04:56:25.943243+0000","last_became_peered":"2024-01-19T04:56:25.943243+0000","last_unstale":"2024-01-19T04:56:25.945347+0000","last_undegraded":"2024-01-19T04:56:25.945347+0000","last_fullsized":"2024-01-19T04:56:25.945347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:38.155889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950160+0000","last_change":"2024-01-19T04:56:24.926841+0000","last_active":"2024-01-19T04:56:25.950160+0000","last_peered":"2024-01-19T04:56:25.950160+0000","last_clean":"2024-01-19T04:56:25.950160+0000","last_became_active":"2024-01-19T04:56:24.926494+0000","last_became_peered":"2024-01-19T04:56:24.926494+0000","last_unstale":"2024-01-19T04:56:25.950160+0000","last_undegraded":"2024-01-19T04:56:25.950160+0000","last_fullsized":"2024-01-19T04:56:25.950160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:17:38.834388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943210+0000","last_change":"2024-01-19T04:56:25.935128+0000","last_active":"2024-01-19T04:56:25.943210+0000","last_peered":"2024-01-19T04:56:25.943210+0000","last_clean":"2024-01-19T04:56:25.943210+0000","last_became_active":"2024-01-19T04:56:25.934889+0000","last_became_peered":"2024-01-19T04:56:25.934889+0000","last_unstale":"2024-01-19T04:56:25.943210+0000","last_undegraded":"2024-01-19T04:56:25.943210+0000","last_fullsized":"2024-01-19T04:56:25.943210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:43:41.276962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.929425+0000","last_change":"2024-01-19T04:56:24.928217+0000","last_active":"2024-01-19T04:56:25.929425+0000","last_peered":"2024-01-19T04:56:25.929425+0000","last_clean":"2024-01-19T04:56:25.929425+0000","last_became_active":"2024-01-19T04:56:24.927573+0000","last_became_peered":"2024-01-19T04:56:24.927573+0000","last_unstale":"2024-01-19T04:56:25.929425+0000","last_undegraded":"2024-01-19T04:56:25.929425+0000","last_fullsized":"2024-01-19T04:56:25.929425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:43:00.441468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944954+0000","last_change":"2024-01-19T04:56:25.943118+0000","last_active":"2024-01-19T04:56:25.944954+0000","last_peered":"2024-01-19T04:56:25.944954+0000","last_clean":"2024-01-19T04:56:25.944954+0000","last_became_active":"2024-01-19T04:56:25.942634+0000","last_became_peered":"2024-01-19T04:56:25.942634+0000","last_unstale":"2024-01-19T04:56:25.944954+0000","last_undegraded":"2024-01-19T04:56:25.944954+0000","last_fullsized":"2024-01-19T04:56:25.944954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:02.466345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926277+0000","last_change":"2024-01-19T04:56:24.932474+0000","last_active":"2024-01-19T04:56:25.926277+0000","last_peered":"2024-01-19T04:56:25.926277+0000","last_clean":"2024-01-19T04:56:25.926277+0000","last_became_active":"2024-01-19T04:56:24.932247+0000","last_became_peered":"2024-01-19T04:56:24.932247+0000","last_unstale":"2024-01-19T04:56:25.926277+0000","last_undegraded":"2024-01-19T04:56:25.926277+0000","last_fullsized":"2024-01-19T04:56:25.926277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:01.460826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944116+0000","last_change":"2024-01-19T04:56:25.942182+0000","last_active":"2024-01-19T04:56:25.944116+0000","last_peered":"2024-01-19T04:56:25.944116+0000","last_clean":"2024-01-19T04:56:25.944116+0000","last_became_active":"2024-01-19T04:56:25.941922+0000","last_became_peered":"2024-01-19T04:56:25.941922+0000","last_unstale":"2024-01-19T04:56:25.944116+0000","last_undegraded":"2024-01-19T04:56:25.944116+0000","last_fullsized":"2024-01-19T04:56:25.944116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:02.088796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954953+0000","last_change":"2024-01-19T04:56:24.926332+0000","last_active":"2024-01-19T04:56:25.954953+0000","last_peered":"2024-01-19T04:56:25.954953+0000","last_clean":"2024-01-19T04:56:25.954953+0000","last_became_active":"2024-01-19T04:56:24.926092+0000","last_became_peered":"2024-01-19T04:56:24.926092+0000","last_unstale":"2024-01-19T04:56:25.954953+0000","last_undegraded":"2024-01-19T04:56:25.954953+0000","last_fullsized":"2024-01-19T04:56:25.954953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:37.150370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.937746+0000","last_change":"2024-01-19T04:56:25.932261+0000","last_active":"2024-01-19T04:56:25.937746+0000","last_peered":"2024-01-19T04:56:25.937746+0000","last_clean":"2024-01-19T04:56:25.937746+0000","last_became_active":"2024-01-19T04:56:25.931981+0000","last_became_peered":"2024-01-19T04:56:25.931981+0000","last_unstale":"2024-01-19T04:56:25.937746+0000","last_undegraded":"2024-01-19T04:56:25.937746+0000","last_fullsized":"2024-01-19T04:56:25.937746+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:59:01.049547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954394+0000","last_change":"2024-01-19T04:56:24.928028+0000","last_active":"2024-01-19T04:56:25.954394+0000","last_peered":"2024-01-19T04:56:25.954394+0000","last_clean":"2024-01-19T04:56:25.954394+0000","last_became_active":"2024-01-19T04:56:24.927732+0000","last_became_peered":"2024-01-19T04:56:24.927732+0000","last_unstale":"2024-01-19T04:56:25.954394+0000","last_undegraded":"2024-01-19T04:56:25.954394+0000","last_fullsized":"2024-01-19T04:56:25.954394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:01.083277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.937120+0000","last_change":"2024-01-19T04:56:25.935252+0000","last_active":"2024-01-19T04:56:25.937120+0000","last_peered":"2024-01-19T04:56:25.937120+0000","last_clean":"2024-01-19T04:56:25.937120+0000","last_became_active":"2024-01-19T04:56:25.934762+0000","last_became_peered":"2024-01-19T04:56:25.934762+0000","last_unstale":"2024-01-19T04:56:25.937120+0000","last_undegraded":"2024-01-19T04:56:25.937120+0000","last_fullsized":"2024-01-19T04:56:25.937120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:02.466345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947994+0000","last_change":"2024-01-19T04:56:24.926522+0000","last_active":"2024-01-19T04:56:25.947994+0000","last_peered":"2024-01-19T04:56:25.947994+0000","last_clean":"2024-01-19T04:56:25.947994+0000","last_became_active":"2024-01-19T04:56:24.926234+0000","last_became_peered":"2024-01-19T04:56:24.926234+0000","last_unstale":"2024-01-19T04:56:25.947994+0000","last_undegraded":"2024-01-19T04:56:25.947994+0000","last_fullsized":"2024-01-19T04:56:25.947994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:51:56.602187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943825+0000","last_change":"2024-01-19T04:56:25.940677+0000","last_active":"2024-01-19T04:56:25.943825+0000","last_peered":"2024-01-19T04:56:25.943825+0000","last_clean":"2024-01-19T04:56:25.943825+0000","last_became_active":"2024-01-19T04:56:25.940078+0000","last_became_peered":"2024-01-19T04:56:25.940078+0000","last_unstale":"2024-01-19T04:56:25.943825+0000","last_undegraded":"2024-01-19T04:56:25.943825+0000","last_fullsized":"2024-01-19T04:56:25.943825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:04.091802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947782+0000","last_change":"2024-01-19T04:56:24.925608+0000","last_active":"2024-01-19T04:56:25.947782+0000","last_peered":"2024-01-19T04:56:25.947782+0000","last_clean":"2024-01-19T04:56:25.947782+0000","last_became_active":"2024-01-19T04:56:24.925121+0000","last_became_peered":"2024-01-19T04:56:24.925121+0000","last_unstale":"2024-01-19T04:56:25.947782+0000","last_undegraded":"2024-01-19T04:56:25.947782+0000","last_fullsized":"2024-01-19T04:56:25.947782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:57:30.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945044+0000","last_change":"2024-01-19T04:56:25.937633+0000","last_active":"2024-01-19T04:56:25.945044+0000","last_peered":"2024-01-19T04:56:25.945044+0000","last_clean":"2024-01-19T04:56:25.945044+0000","last_became_active":"2024-01-19T04:56:25.937362+0000","last_became_peered":"2024-01-19T04:56:25.937362+0000","last_unstale":"2024-01-19T04:56:25.945044+0000","last_undegraded":"2024-01-19T04:56:25.945044+0000","last_fullsized":"2024-01-19T04:56:25.945044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:50:10.958541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.925025+0000","last_change":"2024-01-19T04:56:24.928044+0000","last_active":"2024-01-19T04:56:25.925025+0000","last_peered":"2024-01-19T04:56:25.925025+0000","last_clean":"2024-01-19T04:56:25.925025+0000","last_became_active":"2024-01-19T04:56:24.927759+0000","last_became_peered":"2024-01-19T04:56:24.927759+0000","last_unstale":"2024-01-19T04:56:25.925025+0000","last_undegraded":"2024-01-19T04:56:25.925025+0000","last_fullsized":"2024-01-19T04:56:25.925025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:56:40.404698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947205+0000","last_change":"2024-01-19T04:56:25.943055+0000","last_active":"2024-01-19T04:56:25.947205+0000","last_peered":"2024-01-19T04:56:25.947205+0000","last_clean":"2024-01-19T04:56:25.947205+0000","last_became_active":"2024-01-19T04:56:25.942775+0000","last_became_peered":"2024-01-19T04:56:25.942775+0000","last_unstale":"2024-01-19T04:56:25.947205+0000","last_undegraded":"2024-01-19T04:56:25.947205+0000","last_fullsized":"2024-01-19T04:56:25.947205+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:56:41.410217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946232+0000","last_change":"2024-01-19T04:56:24.926337+0000","last_active":"2024-01-19T04:56:25.946232+0000","last_peered":"2024-01-19T04:56:25.946232+0000","last_clean":"2024-01-19T04:56:25.946232+0000","last_became_active":"2024-01-19T04:56:24.926083+0000","last_became_peered":"2024-01-19T04:56:24.926083+0000","last_unstale":"2024-01-19T04:56:25.946232+0000","last_undegraded":"2024-01-19T04:56:25.946232+0000","last_fullsized":"2024-01-19T04:56:25.946232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:13.511105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.940590+0000","last_change":"2024-01-19T04:56:25.934203+0000","last_active":"2024-01-19T04:56:25.940590+0000","last_peered":"2024-01-19T04:56:25.940590+0000","last_clean":"2024-01-19T04:56:25.940590+0000","last_became_active":"2024-01-19T04:56:25.933933+0000","last_became_peered":"2024-01-19T04:56:25.933933+0000","last_unstale":"2024-01-19T04:56:25.940590+0000","last_undegraded":"2024-01-19T04:56:25.940590+0000","last_fullsized":"2024-01-19T04:56:25.940590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:11.475502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.936853+0000","last_change":"2024-01-19T04:56:24.930127+0000","last_active":"2024-01-19T04:56:25.936853+0000","last_peered":"2024-01-19T04:56:25.936853+0000","last_clean":"2024-01-19T04:56:25.936853+0000","last_became_active":"2024-01-19T04:56:24.929877+0000","last_became_peered":"2024-01-19T04:56:24.929877+0000","last_unstale":"2024-01-19T04:56:25.936853+0000","last_undegraded":"2024-01-19T04:56:25.936853+0000","last_fullsized":"2024-01-19T04:56:25.936853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:13:56.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947294+0000","last_change":"2024-01-19T04:56:25.941008+0000","last_active":"2024-01-19T04:56:25.947294+0000","last_peered":"2024-01-19T04:56:25.947294+0000","last_clean":"2024-01-19T04:56:25.947294+0000","last_became_active":"2024-01-19T04:56:25.940703+0000","last_became_peered":"2024-01-19T04:56:25.940703+0000","last_unstale":"2024-01-19T04:56:25.947294+0000","last_undegraded":"2024-01-19T04:56:25.947294+0000","last_fullsized":"2024-01-19T04:56:25.947294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:31:36.515465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926094+0000","last_change":"2024-01-19T04:56:24.929045+0000","last_active":"2024-01-19T04:56:25.926094+0000","last_peered":"2024-01-19T04:56:25.926094+0000","last_clean":"2024-01-19T04:56:25.926094+0000","last_became_active":"2024-01-19T04:56:24.928761+0000","last_became_peered":"2024-01-19T04:56:24.928761+0000","last_unstale":"2024-01-19T04:56:25.926094+0000","last_undegraded":"2024-01-19T04:56:25.926094+0000","last_fullsized":"2024-01-19T04:56:25.926094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:42:52.025292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950677+0000","last_change":"2024-01-19T04:56:25.946061+0000","last_active":"2024-01-19T04:56:25.950677+0000","last_peered":"2024-01-19T04:56:25.950677+0000","last_clean":"2024-01-19T04:56:25.950677+0000","last_became_active":"2024-01-19T04:56:25.945574+0000","last_became_peered":"2024-01-19T04:56:25.945574+0000","last_unstale":"2024-01-19T04:56:25.950677+0000","last_undegraded":"2024-01-19T04:56:25.950677+0000","last_fullsized":"2024-01-19T04:56:25.950677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:58:19.400385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.924698+0000","last_change":"2024-01-19T04:56:24.930342+0000","last_active":"2024-01-19T04:56:25.924698+0000","last_peered":"2024-01-19T04:56:25.924698+0000","last_clean":"2024-01-19T04:56:25.924698+0000","last_became_active":"2024-01-19T04:56:24.929993+0000","last_became_peered":"2024-01-19T04:56:24.929993+0000","last_unstale":"2024-01-19T04:56:25.924698+0000","last_undegraded":"2024-01-19T04:56:25.924698+0000","last_fullsized":"2024-01-19T04:56:25.924698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:16.685827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946938+0000","last_change":"2024-01-19T04:56:25.938198+0000","last_active":"2024-01-19T04:56:25.946938+0000","last_peered":"2024-01-19T04:56:25.946938+0000","last_clean":"2024-01-19T04:56:25.946938+0000","last_became_active":"2024-01-19T04:56:25.937953+0000","last_became_peered":"2024-01-19T04:56:25.937953+0000","last_unstale":"2024-01-19T04:56:25.946938+0000","last_undegraded":"2024-01-19T04:56:25.946938+0000","last_fullsized":"2024-01-19T04:56:25.946938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:55:17.134541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953468+0000","last_change":"2024-01-19T04:56:24.928922+0000","last_active":"2024-01-19T04:56:25.953468+0000","last_peered":"2024-01-19T04:56:25.953468+0000","last_clean":"2024-01-19T04:56:25.953468+0000","last_became_active":"2024-01-19T04:56:24.928428+0000","last_became_peered":"2024-01-19T04:56:24.928428+0000","last_unstale":"2024-01-19T04:56:25.953468+0000","last_undegraded":"2024-01-19T04:56:25.953468+0000","last_fullsized":"2024-01-19T04:56:25.953468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:10.280240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950784+0000","last_change":"2024-01-19T04:56:25.945424+0000","last_active":"2024-01-19T04:56:25.950784+0000","last_peered":"2024-01-19T04:56:25.950784+0000","last_clean":"2024-01-19T04:56:25.950784+0000","last_became_active":"2024-01-19T04:56:25.945145+0000","last_became_peered":"2024-01-19T04:56:25.945145+0000","last_unstale":"2024-01-19T04:56:25.950784+0000","last_undegraded":"2024-01-19T04:56:25.950784+0000","last_fullsized":"2024-01-19T04:56:25.950784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:24.207122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.923672+0000","last_change":"2024-01-19T04:56:24.929929+0000","last_active":"2024-01-19T04:56:25.923672+0000","last_peered":"2024-01-19T04:56:25.923672+0000","last_clean":"2024-01-19T04:56:25.923672+0000","last_became_active":"2024-01-19T04:56:24.929635+0000","last_became_peered":"2024-01-19T04:56:24.929635+0000","last_unstale":"2024-01-19T04:56:25.923672+0000","last_undegraded":"2024-01-19T04:56:25.923672+0000","last_fullsized":"2024-01-19T04:56:25.923672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:43:00.441468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947580+0000","last_change":"2024-01-19T04:56:25.944802+0000","last_active":"2024-01-19T04:56:25.947580+0000","last_peered":"2024-01-19T04:56:25.947580+0000","last_clean":"2024-01-19T04:56:25.947580+0000","last_became_active":"2024-01-19T04:56:25.944519+0000","last_became_peered":"2024-01-19T04:56:25.944519+0000","last_unstale":"2024-01-19T04:56:25.947580+0000","last_undegraded":"2024-01-19T04:56:25.947580+0000","last_fullsized":"2024-01-19T04:56:25.947580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.928037+0000","last_change":"2024-01-19T04:56:24.928531+0000","last_active":"2024-01-19T04:56:25.928037+0000","last_peered":"2024-01-19T04:56:25.928037+0000","last_clean":"2024-01-19T04:56:25.928037+0000","last_became_active":"2024-01-19T04:56:24.928185+0000","last_became_peered":"2024-01-19T04:56:24.928185+0000","last_unstale":"2024-01-19T04:56:25.928037+0000","last_undegraded":"2024-01-19T04:56:25.928037+0000","last_fullsized":"2024-01-19T04:56:25.928037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:32.417303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947528+0000","last_change":"2024-01-19T04:56:25.943570+0000","last_active":"2024-01-19T04:56:25.947528+0000","last_peered":"2024-01-19T04:56:25.947528+0000","last_clean":"2024-01-19T04:56:25.947528+0000","last_became_active":"2024-01-19T04:56:25.943291+0000","last_became_peered":"2024-01-19T04:56:25.943291+0000","last_unstale":"2024-01-19T04:56:25.947528+0000","last_undegraded":"2024-01-19T04:56:25.947528+0000","last_fullsized":"2024-01-19T04:56:25.947528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:33.422822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.923319+0000","last_change":"2024-01-19T04:56:24.920947+0000","last_active":"2024-01-19T04:56:25.923319+0000","last_peered":"2024-01-19T04:56:25.923319+0000","last_clean":"2024-01-19T04:56:25.923319+0000","last_became_active":"2024-01-19T04:56:24.920655+0000","last_became_peered":"2024-01-19T04:56:24.920655+0000","last_unstale":"2024-01-19T04:56:25.923319+0000","last_undegraded":"2024-01-19T04:56:25.923319+0000","last_fullsized":"2024-01-19T04:56:25.923319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:01.083277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.942087+0000","last_change":"2024-01-19T04:56:25.940355+0000","last_active":"2024-01-19T04:56:25.942087+0000","last_peered":"2024-01-19T04:56:25.942087+0000","last_clean":"2024-01-19T04:56:25.942087+0000","last_became_active":"2024-01-19T04:56:25.940092+0000","last_became_peered":"2024-01-19T04:56:25.940092+0000","last_unstale":"2024-01-19T04:56:25.942087+0000","last_undegraded":"2024-01-19T04:56:25.942087+0000","last_fullsized":"2024-01-19T04:56:25.942087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:04.091802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.927530+0000","last_change":"2024-01-19T04:56:24.927768+0000","last_active":"2024-01-19T04:56:25.927530+0000","last_peered":"2024-01-19T04:56:25.927530+0000","last_clean":"2024-01-19T04:56:25.927530+0000","last_became_active":"2024-01-19T04:56:24.927215+0000","last_became_peered":"2024-01-19T04:56:24.927215+0000","last_unstale":"2024-01-19T04:56:25.927530+0000","last_undegraded":"2024-01-19T04:56:25.927530+0000","last_fullsized":"2024-01-19T04:56:25.927530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:13.511105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947850+0000","last_change":"2024-01-19T04:56:25.946278+0000","last_active":"2024-01-19T04:56:25.947850+0000","last_peered":"2024-01-19T04:56:25.947850+0000","last_clean":"2024-01-19T04:56:25.947850+0000","last_became_active":"2024-01-19T04:56:25.946014+0000","last_became_peered":"2024-01-19T04:56:25.946014+0000","last_unstale":"2024-01-19T04:56:25.947850+0000","last_undegraded":"2024-01-19T04:56:25.947850+0000","last_fullsized":"2024-01-19T04:56:25.947850+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:25:48.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926232+0000","last_change":"2024-01-19T04:56:24.929537+0000","last_active":"2024-01-19T04:56:25.926232+0000","last_peered":"2024-01-19T04:56:25.926232+0000","last_clean":"2024-01-19T04:56:25.926232+0000","last_became_active":"2024-01-19T04:56:24.929276+0000","last_became_peered":"2024-01-19T04:56:24.929276+0000","last_unstale":"2024-01-19T04:56:25.926232+0000","last_undegraded":"2024-01-19T04:56:25.926232+0000","last_fullsized":"2024-01-19T04:56:25.926232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:07:53.445003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944580+0000","last_change":"2024-01-19T04:56:25.941715+0000","last_active":"2024-01-19T04:56:25.944580+0000","last_peered":"2024-01-19T04:56:25.944580+0000","last_clean":"2024-01-19T04:56:25.944580+0000","last_became_active":"2024-01-19T04:56:25.941460+0000","last_became_peered":"2024-01-19T04:56:25.941460+0000","last_unstale":"2024-01-19T04:56:25.944580+0000","last_undegraded":"2024-01-19T04:56:25.944580+0000","last_fullsized":"2024-01-19T04:56:25.944580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:27:22.525627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926873+0000","last_change":"2024-01-19T04:56:24.929140+0000","last_active":"2024-01-19T04:56:25.926873+0000","last_peered":"2024-01-19T04:56:25.926873+0000","last_clean":"2024-01-19T04:56:25.926873+0000","last_became_active":"2024-01-19T04:56:24.928854+0000","last_became_peered":"2024-01-19T04:56:24.928854+0000","last_unstale":"2024-01-19T04:56:25.926873+0000","last_undegraded":"2024-01-19T04:56:25.926873+0000","last_fullsized":"2024-01-19T04:56:25.926873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:48.029021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953257+0000","last_change":"2024-01-19T04:56:25.949297+0000","last_active":"2024-01-19T04:56:25.953257+0000","last_peered":"2024-01-19T04:56:25.953257+0000","last_clean":"2024-01-19T04:56:25.953257+0000","last_became_active":"2024-01-19T04:56:25.949004+0000","last_became_peered":"2024-01-19T04:56:25.949004+0000","last_unstale":"2024-01-19T04:56:25.953257+0000","last_undegraded":"2024-01-19T04:56:25.953257+0000","last_fullsized":"2024-01-19T04:56:25.953257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:43:41.276962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.937346+0000","last_change":"2024-01-19T04:56:24.927874+0000","last_active":"2024-01-19T04:56:25.937346+0000","last_peered":"2024-01-19T04:56:25.937346+0000","last_clean":"2024-01-19T04:56:25.937346+0000","last_became_active":"2024-01-19T04:56:24.927592+0000","last_became_peered":"2024-01-19T04:56:24.927592+0000","last_unstale":"2024-01-19T04:56:25.937346+0000","last_undegraded":"2024-01-19T04:56:25.937346+0000","last_fullsized":"2024-01-19T04:56:25.937346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:27:21.520108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-19T04:56:24.924463+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:56:24.924463+0000","last_peered":"2024-01-19T04:56:24.924463+0000","last_clean":"2024-01-19T04:56:24.924463+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:56:24.924463+0000","last_undegraded":"2024-01-19T04:56:24.924463+0000","last_fullsized":"2024-01-19T04:56:24.924463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":736065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":4.7800000000000002,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.629},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.79400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.439},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.36599999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.41399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63}]},{"osd":3,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.314},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.437}]},{"osd":7,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.504}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.373}]},{"osd":3,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.39700000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.626},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.65300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.251},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.23899999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.27000000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.624},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":735975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.223},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.379}]},{"osd":2,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.252},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.505},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:56:28.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:56:29.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.064+0000 7f119221e640 1 Processor -- start 2024-01-19T04:56:29.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.064+0000 7f119221e640 1 -- start start 2024-01-19T04:56:29.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f119221e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f118c15eb70 0x7f118c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f119221e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c06bf30 con 0x7f118c15d8a0 2024-01-19T04:56:29.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f119221e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0cb9e0 con 0x7f118c0cca00 2024-01-19T04:56:29.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f119221e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c0cbcc0 con 0x7f118c15eb70 2024-01-19T04:56:29.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f118c15d8a0 0x7f118c06b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39638/0 (socket says 172.21.15.31:39638) 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118b7fe640 1 -- 172.21.15.31:0/2144073079 learned_addr learned my addr 172.21.15.31:0/2144073079 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053637257 0 0) 0x7f118c06bf30 con 0x7f118c15d8a0 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1174003520 con 0x7f118c15d8a0 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118affd640 1 --2- 172.21.15.31:0/2144073079 >> v2:172.21.15.137:3300/0 conn(0x7f118c15eb70 0x7f118c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704178164 0 0) 0x7f1174003520 con 0x7f118c15d8a0 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 >> v2:172.21.15.137:3300/0 conn(0x7f118c15eb70 msgr2=0x7f118c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 --2- 172.21.15.31:0/2144073079 >> v2:172.21.15.137:3300/0 conn(0x7f118c15eb70 0x7f118c0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 >> v1:172.21.15.137:6789/0 conn(0x7f118c0cca00 legacy=0x7f118c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c0cbfa0 con 0x7f118c15d8a0 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.065+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f117c001670 con 0x7f118c15d8a0 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 >> v1:172.21.15.31:6789/0 conn(0x7f118c15d8a0 legacy=0x7f118c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118a7fc640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174003970 0x7f1174003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118a7fc640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11740047c0 0x7f1174004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118a7fc640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174005180 0x7f117400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0cbcc0 con 0x7f1174003970 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117400c120 con 0x7f11740047c0 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117400c2e0 con 0x7f1174005180 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118b7fe640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174003970 0x7f1174003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118affd640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11740047c0 0x7f1174004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118bfff640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174005180 0x7f117400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118b7fe640 1 -- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174005180 msgr2=0x7f117400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118b7fe640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174005180 0x7f117400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118b7fe640 1 -- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11740047c0 msgr2=0x7f1174004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118b7fe640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11740047c0 0x7f1174004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118b7fe640 1 -- 172.21.15.31:0/2144073079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174003520 con 0x7f1174003970 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118affd640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11740047c0 0x7f1174004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.066+0000 7f118bfff640 1 -- 172.21.15.31:0/2144073079 reap_dead start 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f118b7fe640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174003970 0x7f1174003d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c240 tx=0x7f11740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19995f47e217beb9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c005ad0 con 0x7f1174003970 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c005c30 con 0x7f1174003970 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f118a7fc640 1 -- 172.21.15.31:0/2144073079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c02caf0 con 0x7f1174003970 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f119221e640 1 -- 172.21.15.31:0/2144073079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174003970 msgr2=0x7f1174003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f119221e640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174003970 0x7f1174003d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c240 tx=0x7f11740057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f119221e640 1 -- 172.21.15.31:0/2144073079 shutdown_connections 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f119221e640 1 --2- 172.21.15.31:0/2144073079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174003970 0x7f1174003d90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f119221e640 1 -- 172.21.15.31:0/2144073079 >> 172.21.15.31:0/2144073079 conn(0x7f118c06abe0 msgr2=0x7f118c066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f119221e640 1 -- 172.21.15.31:0/2144073079 shutdown_connections 2024-01-19T04:56:29.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.067+0000 7f119221e640 1 -- 172.21.15.31:0/2144073079 wait complete. 2024-01-19T04:56:29.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.068+0000 7f119221e640 1 Processor -- start 2024-01-19T04:56:29.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f119221e640 1 -- start start 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f119221e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174003970 0x7f118c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f119221e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1600b0 0x7f118c1488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f119221e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f118c160530 0x7f118c148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f119221e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0c7ae0 con 0x7f118c1600b0 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f119221e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c160a20 con 0x7f118c160530 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f119221e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c160d00 con 0x7f1174003970 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f118affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1600b0 0x7f118c1488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f118affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1600b0 0x7f118c1488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58278/0 (socket says 172.21.15.31:58278) 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.069+0000 7f118affd640 1 -- 172.21.15.31:0/764146378 learned_addr learned my addr 172.21.15.31:0/764146378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118bfff640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f118c160530 0x7f118c148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118affd640 1 -- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174003970 msgr2=0x7f118c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118b7fe640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174003970 0x7f118c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118affd640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174003970 0x7f118c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118affd640 1 -- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f118c160530 msgr2=0x7f118c148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118affd640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f118c160530 0x7f118c148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118affd640 1 -- 172.21.15.31:0/764146378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c160fe0 con 0x7f118c1600b0 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118bfff640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f118c160530 0x7f118c148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118b7fe640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174003970 0x7f118c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f118affd640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1600b0 0x7f118c1488d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11800114e0 tx=0x7f11800119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f1173fff640 1 -- 172.21.15.31:0/764146378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180015870 con 0x7f118c1600b0 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f1173fff640 1 -- 172.21.15.31:0/764146378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11800159d0 con 0x7f118c1600b0 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f1173fff640 1 -- 172.21.15.31:0/764146378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180015d80 con 0x7f118c1600b0 2024-01-19T04:56:29.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c1612c0 con 0x7f118c1600b0 2024-01-19T04:56:29.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.070+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c0b4630 con 0x7f118c1600b0 2024-01-19T04:56:29.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.071+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0696e0 con 0x7f118c1600b0 2024-01-19T04:56:29.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.072+0000 7f1173fff640 1 -- 172.21.15.31:0/764146378 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f118001d400 con 0x7f118c1600b0 2024-01-19T04:56:29.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.072+0000 7f1173fff640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f117806aed0 0x7f117806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.072+0000 7f1173fff640 1 -- 172.21.15.31:0/764146378 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1180089d10 con 0x7f118c1600b0 2024-01-19T04:56:29.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.072+0000 7f118b7fe640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f117806aed0 0x7f117806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.073+0000 7f118b7fe640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f117806aed0 0x7f117806d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f117c034000 tx=0x7f117c009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.076+0000 7f1173fff640 1 -- 172.21.15.31:0/764146378 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1180054be0 con 0x7f118c1600b0 2024-01-19T04:56:29.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.252+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f118c15dc80 con 0x7f117806aed0 2024-01-19T04:56:29.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.260+0000 7f1173fff640 1 -- 172.21.15.31:0/764146378 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289113 (secure 0 0 0) 0x7f117c0014b0 con 0x7f117806aed0 2024-01-19T04:56:29.263 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:56:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f117806aed0 msgr2=0x7f117806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f117806aed0 0x7f117806d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f117c034000 tx=0x7f117c009030 comp rx=0 tx=0).stop 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1600b0 msgr2=0x7f118c1488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1600b0 0x7f118c1488d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11800114e0 tx=0x7f11800119a0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 shutdown_connections 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f117806aed0 0x7f117806d390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f118c160530 0x7f118c148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1600b0 0x7f118c1488d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 --2- 172.21.15.31:0/764146378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1174003970 0x7f118c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 >> 172.21.15.31:0/764146378 conn(0x7f118c06abe0 msgr2=0x7f118c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 shutdown_connections 2024-01-19T04:56:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.263+0000 7f119221e640 1 -- 172.21.15.31:0/764146378 wait complete. 2024-01-19T04:56:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-01-19T04:56:27.576917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220284,"kb_used_data":4108,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754436,"statfs":{"total":773094113280,"available":772868542464,"internally_reserved":0,"allocated":4206592,"data_stored":2342003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000917"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943722+0000","last_change":"2024-01-19T04:56:25.938864+0000","last_active":"2024-01-19T04:56:25.943722+0000","last_peered":"2024-01-19T04:56:25.943722+0000","last_clean":"2024-01-19T04:56:25.943722+0000","last_became_active":"2024-01-19T04:56:25.938572+0000","last_became_peered":"2024-01-19T04:56:25.938572+0000","last_unstale":"2024-01-19T04:56:25.943722+0000","last_undegraded":"2024-01-19T04:56:25.943722+0000","last_fullsized":"2024-01-19T04:56:25.943722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:14.516624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944322+0000","last_change":"2024-01-19T04:56:25.941620+0000","last_active":"2024-01-19T04:56:25.944322+0000","last_peered":"2024-01-19T04:56:25.944322+0000","last_clean":"2024-01-19T04:56:25.944322+0000","last_became_active":"2024-01-19T04:56:25.941100+0000","last_became_peered":"2024-01-19T04:56:25.941100+0000","last_unstale":"2024-01-19T04:56:25.944322+0000","last_undegraded":"2024-01-19T04:56:25.944322+0000","last_fullsized":"2024-01-19T04:56:25.944322+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:13:42.229617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950804+0000","last_change":"2024-01-19T04:56:25.945484+0000","last_active":"2024-01-19T04:56:25.950804+0000","last_peered":"2024-01-19T04:56:25.950804+0000","last_clean":"2024-01-19T04:56:25.950804+0000","last_became_active":"2024-01-19T04:56:25.944935+0000","last_became_peered":"2024-01-19T04:56:25.944935+0000","last_unstale":"2024-01-19T04:56:25.950804+0000","last_undegraded":"2024-01-19T04:56:25.950804+0000","last_fullsized":"2024-01-19T04:56:25.950804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:58:19.400385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943888+0000","last_change":"2024-01-19T04:56:25.941103+0000","last_active":"2024-01-19T04:56:25.943888+0000","last_peered":"2024-01-19T04:56:25.943888+0000","last_clean":"2024-01-19T04:56:25.943888+0000","last_became_active":"2024-01-19T04:56:25.940387+0000","last_became_peered":"2024-01-19T04:56:25.940387+0000","last_unstale":"2024-01-19T04:56:25.943888+0000","last_undegraded":"2024-01-19T04:56:25.943888+0000","last_fullsized":"2024-01-19T04:56:25.943888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946892+0000","last_change":"2024-01-19T04:56:25.940429+0000","last_active":"2024-01-19T04:56:25.946892+0000","last_peered":"2024-01-19T04:56:25.946892+0000","last_clean":"2024-01-19T04:56:25.946892+0000","last_became_active":"2024-01-19T04:56:25.940179+0000","last_became_peered":"2024-01-19T04:56:25.940179+0000","last_unstale":"2024-01-19T04:56:25.946892+0000","last_undegraded":"2024-01-19T04:56:25.946892+0000","last_fullsized":"2024-01-19T04:56:25.946892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:00:17.149689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943679+0000","last_change":"2024-01-19T04:56:25.935733+0000","last_active":"2024-01-19T04:56:25.943679+0000","last_peered":"2024-01-19T04:56:25.943679+0000","last_clean":"2024-01-19T04:56:25.943679+0000","last_became_active":"2024-01-19T04:56:25.935422+0000","last_became_peered":"2024-01-19T04:56:25.935422+0000","last_unstale":"2024-01-19T04:56:25.943679+0000","last_undegraded":"2024-01-19T04:56:25.943679+0000","last_fullsized":"2024-01-19T04:56:25.943679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:12:35.815320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950700+0000","last_change":"2024-01-19T04:56:25.945826+0000","last_active":"2024-01-19T04:56:25.950700+0000","last_peered":"2024-01-19T04:56:25.950700+0000","last_clean":"2024-01-19T04:56:25.950700+0000","last_became_active":"2024-01-19T04:56:25.945218+0000","last_became_peered":"2024-01-19T04:56:25.945218+0000","last_unstale":"2024-01-19T04:56:25.950700+0000","last_undegraded":"2024-01-19T04:56:25.950700+0000","last_fullsized":"2024-01-19T04:56:25.950700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:24.207122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944932+0000","last_change":"2024-01-19T04:56:25.942901+0000","last_active":"2024-01-19T04:56:25.944932+0000","last_peered":"2024-01-19T04:56:25.944932+0000","last_clean":"2024-01-19T04:56:25.944932+0000","last_became_active":"2024-01-19T04:56:25.942291+0000","last_became_peered":"2024-01-19T04:56:25.942291+0000","last_unstale":"2024-01-19T04:56:25.944932+0000","last_undegraded":"2024-01-19T04:56:25.944932+0000","last_fullsized":"2024-01-19T04:56:25.944932+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950647+0000","last_change":"2024-01-19T04:56:25.944837+0000","last_active":"2024-01-19T04:56:25.950647+0000","last_peered":"2024-01-19T04:56:25.950647+0000","last_clean":"2024-01-19T04:56:25.950647+0000","last_became_active":"2024-01-19T04:56:25.944549+0000","last_became_peered":"2024-01-19T04:56:25.944549+0000","last_unstale":"2024-01-19T04:56:25.950647+0000","last_undegraded":"2024-01-19T04:56:25.950647+0000","last_fullsized":"2024-01-19T04:56:25.950647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:11.285759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953835+0000","last_change":"2024-01-19T04:56:25.950327+0000","last_active":"2024-01-19T04:56:25.953835+0000","last_peered":"2024-01-19T04:56:25.953835+0000","last_clean":"2024-01-19T04:56:25.953835+0000","last_became_active":"2024-01-19T04:56:25.950027+0000","last_became_peered":"2024-01-19T04:56:25.950027+0000","last_unstale":"2024-01-19T04:56:25.953835+0000","last_undegraded":"2024-01-19T04:56:25.953835+0000","last_fullsized":"2024-01-19T04:56:25.953835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:49.034540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950610+0000","last_change":"2024-01-19T04:56:25.945557+0000","last_active":"2024-01-19T04:56:25.950610+0000","last_peered":"2024-01-19T04:56:25.950610+0000","last_clean":"2024-01-19T04:56:25.950610+0000","last_became_active":"2024-01-19T04:56:25.945267+0000","last_became_peered":"2024-01-19T04:56:25.945267+0000","last_unstale":"2024-01-19T04:56:25.950610+0000","last_undegraded":"2024-01-19T04:56:25.950610+0000","last_fullsized":"2024-01-19T04:56:25.950610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:49.034540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950894+0000","last_change":"2024-01-19T04:56:25.946677+0000","last_active":"2024-01-19T04:56:25.950894+0000","last_peered":"2024-01-19T04:56:25.950894+0000","last_clean":"2024-01-19T04:56:25.950894+0000","last_became_active":"2024-01-19T04:56:25.946372+0000","last_became_peered":"2024-01-19T04:56:25.946372+0000","last_unstale":"2024-01-19T04:56:25.950894+0000","last_undegraded":"2024-01-19T04:56:25.950894+0000","last_fullsized":"2024-01-19T04:56:25.950894+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:30:16.180514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947391+0000","last_change":"2024-01-19T04:56:25.945260+0000","last_active":"2024-01-19T04:56:25.947391+0000","last_peered":"2024-01-19T04:56:25.947391+0000","last_clean":"2024-01-19T04:56:25.947391+0000","last_became_active":"2024-01-19T04:56:25.945002+0000","last_became_peered":"2024-01-19T04:56:25.945002+0000","last_unstale":"2024-01-19T04:56:25.947391+0000","last_undegraded":"2024-01-19T04:56:25.947391+0000","last_fullsized":"2024-01-19T04:56:25.947391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:57:31.321604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944622+0000","last_change":"2024-01-19T04:56:25.939505+0000","last_active":"2024-01-19T04:56:25.944622+0000","last_peered":"2024-01-19T04:56:25.944622+0000","last_clean":"2024-01-19T04:56:25.944622+0000","last_became_active":"2024-01-19T04:56:25.938996+0000","last_became_peered":"2024-01-19T04:56:25.938996+0000","last_unstale":"2024-01-19T04:56:25.944622+0000","last_undegraded":"2024-01-19T04:56:25.944622+0000","last_fullsized":"2024-01-19T04:56:25.944622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:30:16.180514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945007+0000","last_change":"2024-01-19T04:56:25.940633+0000","last_active":"2024-01-19T04:56:25.945007+0000","last_peered":"2024-01-19T04:56:25.945007+0000","last_clean":"2024-01-19T04:56:25.945007+0000","last_became_active":"2024-01-19T04:56:25.940373+0000","last_became_peered":"2024-01-19T04:56:25.940373+0000","last_unstale":"2024-01-19T04:56:25.945007+0000","last_undegraded":"2024-01-19T04:56:25.945007+0000","last_fullsized":"2024-01-19T04:56:25.945007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:49.034540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953289+0000","last_change":"2024-01-19T04:56:25.950156+0000","last_active":"2024-01-19T04:56:25.953289+0000","last_peered":"2024-01-19T04:56:25.953289+0000","last_clean":"2024-01-19T04:56:25.953289+0000","last_became_active":"2024-01-19T04:56:25.949814+0000","last_became_peered":"2024-01-19T04:56:25.949814+0000","last_unstale":"2024-01-19T04:56:25.953289+0000","last_undegraded":"2024-01-19T04:56:25.953289+0000","last_fullsized":"2024-01-19T04:56:25.953289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:12:35.815320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946372+0000","last_change":"2024-01-19T04:56:25.944476+0000","last_active":"2024-01-19T04:56:25.946372+0000","last_peered":"2024-01-19T04:56:25.946372+0000","last_clean":"2024-01-19T04:56:25.946372+0000","last_became_active":"2024-01-19T04:56:25.943852+0000","last_became_peered":"2024-01-19T04:56:25.943852+0000","last_unstale":"2024-01-19T04:56:25.946372+0000","last_undegraded":"2024-01-19T04:56:25.946372+0000","last_fullsized":"2024-01-19T04:56:25.946372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:27.173131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954125+0000","last_change":"2024-01-19T04:56:25.948730+0000","last_active":"2024-01-19T04:56:25.954125+0000","last_peered":"2024-01-19T04:56:25.954125+0000","last_clean":"2024-01-19T04:56:25.954125+0000","last_became_active":"2024-01-19T04:56:25.948458+0000","last_became_peered":"2024-01-19T04:56:25.948458+0000","last_unstale":"2024-01-19T04:56:25.954125+0000","last_undegraded":"2024-01-19T04:56:25.954125+0000","last_fullsized":"2024-01-19T04:56:25.954125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:22:02.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.948489+0000","last_change":"2024-01-19T04:56:25.946921+0000","last_active":"2024-01-19T04:56:25.948489+0000","last_peered":"2024-01-19T04:56:25.948489+0000","last_clean":"2024-01-19T04:56:25.948489+0000","last_became_active":"2024-01-19T04:56:25.946662+0000","last_became_peered":"2024-01-19T04:56:25.946662+0000","last_unstale":"2024-01-19T04:56:25.948489+0000","last_undegraded":"2024-01-19T04:56:25.948489+0000","last_fullsized":"2024-01-19T04:56:25.948489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:06:15.316666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947044+0000","last_change":"2024-01-19T04:56:25.942039+0000","last_active":"2024-01-19T04:56:25.947044+0000","last_peered":"2024-01-19T04:56:25.947044+0000","last_clean":"2024-01-19T04:56:25.947044+0000","last_became_active":"2024-01-19T04:56:25.941715+0000","last_became_peered":"2024-01-19T04:56:25.941715+0000","last_unstale":"2024-01-19T04:56:25.947044+0000","last_undegraded":"2024-01-19T04:56:25.947044+0000","last_fullsized":"2024-01-19T04:56:25.947044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:28:21.687396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.939397+0000","last_change":"2024-01-19T04:56:25.934401+0000","last_active":"2024-01-19T04:56:25.939397+0000","last_peered":"2024-01-19T04:56:25.939397+0000","last_clean":"2024-01-19T04:56:25.939397+0000","last_became_active":"2024-01-19T04:56:25.934134+0000","last_became_peered":"2024-01-19T04:56:25.934134+0000","last_unstale":"2024-01-19T04:56:25.939397+0000","last_undegraded":"2024-01-19T04:56:25.939397+0000","last_fullsized":"2024-01-19T04:56:25.939397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:13:57.205481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944421+0000","last_change":"2024-01-19T04:56:25.942496+0000","last_active":"2024-01-19T04:56:25.944421+0000","last_peered":"2024-01-19T04:56:25.944421+0000","last_clean":"2024-01-19T04:56:25.944421+0000","last_became_active":"2024-01-19T04:56:25.942202+0000","last_became_peered":"2024-01-19T04:56:25.942202+0000","last_unstale":"2024-01-19T04:56:25.944421+0000","last_undegraded":"2024-01-19T04:56:25.944421+0000","last_fullsized":"2024-01-19T04:56:25.944421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:02.088796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.933743+0000","last_change":"2024-01-19T04:56:25.932844+0000","last_active":"2024-01-19T04:56:25.933743+0000","last_peered":"2024-01-19T04:56:25.933743+0000","last_clean":"2024-01-19T04:56:25.933743+0000","last_became_active":"2024-01-19T04:56:25.932606+0000","last_became_peered":"2024-01-19T04:56:25.932606+0000","last_unstale":"2024-01-19T04:56:25.933743+0000","last_undegraded":"2024-01-19T04:56:25.933743+0000","last_fullsized":"2024-01-19T04:56:25.933743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:17.691346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.942752+0000","last_change":"2024-01-19T04:56:25.938490+0000","last_active":"2024-01-19T04:56:25.942752+0000","last_peered":"2024-01-19T04:56:25.942752+0000","last_clean":"2024-01-19T04:56:25.942752+0000","last_became_active":"2024-01-19T04:56:25.938151+0000","last_became_peered":"2024-01-19T04:56:25.938151+0000","last_unstale":"2024-01-19T04:56:25.942752+0000","last_undegraded":"2024-01-19T04:56:25.942752+0000","last_fullsized":"2024-01-19T04:56:25.942752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:27.173131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945652+0000","last_change":"2024-01-19T04:56:25.943682+0000","last_active":"2024-01-19T04:56:25.945652+0000","last_peered":"2024-01-19T04:56:25.945652+0000","last_clean":"2024-01-19T04:56:25.945652+0000","last_became_active":"2024-01-19T04:56:25.943356+0000","last_became_peered":"2024-01-19T04:56:25.943356+0000","last_unstale":"2024-01-19T04:56:25.945652+0000","last_undegraded":"2024-01-19T04:56:25.945652+0000","last_fullsized":"2024-01-19T04:56:25.945652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:27.173131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944300+0000","last_change":"2024-01-19T04:56:25.941402+0000","last_active":"2024-01-19T04:56:25.944300+0000","last_peered":"2024-01-19T04:56:25.944300+0000","last_clean":"2024-01-19T04:56:25.944300+0000","last_became_active":"2024-01-19T04:56:25.940766+0000","last_became_peered":"2024-01-19T04:56:25.940766+0000","last_unstale":"2024-01-19T04:56:25.944300+0000","last_undegraded":"2024-01-19T04:56:25.944300+0000","last_fullsized":"2024-01-19T04:56:25.944300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:43:57.779892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944180+0000","last_change":"2024-01-19T04:56:25.941547+0000","last_active":"2024-01-19T04:56:25.944180+0000","last_peered":"2024-01-19T04:56:25.944180+0000","last_clean":"2024-01-19T04:56:25.944180+0000","last_became_active":"2024-01-19T04:56:25.940049+0000","last_became_peered":"2024-01-19T04:56:25.940049+0000","last_unstale":"2024-01-19T04:56:25.944180+0000","last_undegraded":"2024-01-19T04:56:25.944180+0000","last_fullsized":"2024-01-19T04:56:25.944180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:02.466345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.931653+0000","last_change":"2024-01-19T04:56:25.929826+0000","last_active":"2024-01-19T04:56:25.931653+0000","last_peered":"2024-01-19T04:56:25.931653+0000","last_clean":"2024-01-19T04:56:25.931653+0000","last_became_active":"2024-01-19T04:56:25.929467+0000","last_became_peered":"2024-01-19T04:56:25.929467+0000","last_unstale":"2024-01-19T04:56:25.931653+0000","last_undegraded":"2024-01-19T04:56:25.931653+0000","last_fullsized":"2024-01-19T04:56:25.931653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944464+0000","last_change":"2024-01-19T04:56:25.942031+0000","last_active":"2024-01-19T04:56:25.944464+0000","last_peered":"2024-01-19T04:56:25.944464+0000","last_clean":"2024-01-19T04:56:25.944464+0000","last_became_active":"2024-01-19T04:56:25.941785+0000","last_became_peered":"2024-01-19T04:56:25.941785+0000","last_unstale":"2024-01-19T04:56:25.944464+0000","last_undegraded":"2024-01-19T04:56:25.944464+0000","last_fullsized":"2024-01-19T04:56:25.944464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:14.516624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945800+0000","last_change":"2024-01-19T04:56:25.943449+0000","last_active":"2024-01-19T04:56:25.945800+0000","last_peered":"2024-01-19T04:56:25.945800+0000","last_clean":"2024-01-19T04:56:25.945800+0000","last_became_active":"2024-01-19T04:56:25.943195+0000","last_became_peered":"2024-01-19T04:56:25.943195+0000","last_unstale":"2024-01-19T04:56:25.945800+0000","last_undegraded":"2024-01-19T04:56:25.945800+0000","last_fullsized":"2024-01-19T04:56:25.945800+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:38.155889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950749+0000","last_change":"2024-01-19T04:56:25.948337+0000","last_active":"2024-01-19T04:56:25.950749+0000","last_peered":"2024-01-19T04:56:25.950749+0000","last_clean":"2024-01-19T04:56:25.950749+0000","last_became_active":"2024-01-19T04:56:25.947932+0000","last_became_peered":"2024-01-19T04:56:25.947932+0000","last_unstale":"2024-01-19T04:56:25.950749+0000","last_undegraded":"2024-01-19T04:56:25.950749+0000","last_fullsized":"2024-01-19T04:56:25.950749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:43:01.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943425+0000","last_change":"2024-01-19T04:56:25.941922+0000","last_active":"2024-01-19T04:56:25.943425+0000","last_peered":"2024-01-19T04:56:25.943425+0000","last_clean":"2024-01-19T04:56:25.943425+0000","last_became_active":"2024-01-19T04:56:25.941636+0000","last_became_peered":"2024-01-19T04:56:25.941636+0000","last_unstale":"2024-01-19T04:56:25.943425+0000","last_undegraded":"2024-01-19T04:56:25.943425+0000","last_fullsized":"2024-01-19T04:56:25.943425+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:04.091802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.933743+0000","last_change":"2024-01-19T04:56:25.932234+0000","last_active":"2024-01-19T04:56:25.933743+0000","last_peered":"2024-01-19T04:56:25.933743+0000","last_clean":"2024-01-19T04:56:25.933743+0000","last_became_active":"2024-01-19T04:56:25.931978+0000","last_became_peered":"2024-01-19T04:56:25.931978+0000","last_unstale":"2024-01-19T04:56:25.933743+0000","last_undegraded":"2024-01-19T04:56:25.933743+0000","last_fullsized":"2024-01-19T04:56:25.933743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:59:03.271823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953477+0000","last_change":"2024-01-19T04:56:24.926873+0000","last_active":"2024-01-19T04:56:25.953477+0000","last_peered":"2024-01-19T04:56:25.953477+0000","last_clean":"2024-01-19T04:56:25.953477+0000","last_became_active":"2024-01-19T04:56:24.926598+0000","last_became_peered":"2024-01-19T04:56:24.926598+0000","last_unstale":"2024-01-19T04:56:25.953477+0000","last_undegraded":"2024-01-19T04:56:25.953477+0000","last_fullsized":"2024-01-19T04:56:25.953477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:33:46.879094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944089+0000","last_change":"2024-01-19T04:56:25.942560+0000","last_active":"2024-01-19T04:56:25.944089+0000","last_peered":"2024-01-19T04:56:25.944089+0000","last_clean":"2024-01-19T04:56:25.944089+0000","last_became_active":"2024-01-19T04:56:25.942313+0000","last_became_peered":"2024-01-19T04:56:25.942313+0000","last_unstale":"2024-01-19T04:56:25.944089+0000","last_undegraded":"2024-01-19T04:56:25.944089+0000","last_fullsized":"2024-01-19T04:56:25.944089+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:11.285759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.949143+0000","last_change":"2024-01-19T04:56:24.930063+0000","last_active":"2024-01-19T04:56:25.949143+0000","last_peered":"2024-01-19T04:56:25.949143+0000","last_clean":"2024-01-19T04:56:25.949143+0000","last_became_active":"2024-01-19T04:56:24.929785+0000","last_became_peered":"2024-01-19T04:56:24.929785+0000","last_unstale":"2024-01-19T04:56:25.949143+0000","last_undegraded":"2024-01-19T04:56:25.949143+0000","last_fullsized":"2024-01-19T04:56:25.949143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:10.280240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953260+0000","last_change":"2024-01-19T04:56:25.950148+0000","last_active":"2024-01-19T04:56:25.953260+0000","last_peered":"2024-01-19T04:56:25.953260+0000","last_clean":"2024-01-19T04:56:25.953260+0000","last_became_active":"2024-01-19T04:56:25.949760+0000","last_became_peered":"2024-01-19T04:56:25.949760+0000","last_unstale":"2024-01-19T04:56:25.953260+0000","last_undegraded":"2024-01-19T04:56:25.953260+0000","last_fullsized":"2024-01-19T04:56:25.953260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:33.422822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954694+0000","last_change":"2024-01-19T04:56:24.927145+0000","last_active":"2024-01-19T04:56:25.954694+0000","last_peered":"2024-01-19T04:56:25.954694+0000","last_clean":"2024-01-19T04:56:25.954694+0000","last_became_active":"2024-01-19T04:56:24.926862+0000","last_became_peered":"2024-01-19T04:56:24.926862+0000","last_unstale":"2024-01-19T04:56:25.954694+0000","last_undegraded":"2024-01-19T04:56:25.954694+0000","last_fullsized":"2024-01-19T04:56:25.954694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:03.086283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944129+0000","last_change":"2024-01-19T04:56:25.938848+0000","last_active":"2024-01-19T04:56:25.944129+0000","last_peered":"2024-01-19T04:56:25.944129+0000","last_clean":"2024-01-19T04:56:25.944129+0000","last_became_active":"2024-01-19T04:56:25.938508+0000","last_became_peered":"2024-01-19T04:56:25.938508+0000","last_unstale":"2024-01-19T04:56:25.944129+0000","last_undegraded":"2024-01-19T04:56:25.944129+0000","last_fullsized":"2024-01-19T04:56:25.944129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:17.691346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947135+0000","last_change":"2024-01-19T04:56:24.931702+0000","last_active":"2024-01-19T04:56:25.947135+0000","last_peered":"2024-01-19T04:56:25.947135+0000","last_clean":"2024-01-19T04:56:25.947135+0000","last_became_active":"2024-01-19T04:56:24.931369+0000","last_became_peered":"2024-01-19T04:56:24.931369+0000","last_unstale":"2024-01-19T04:56:25.947135+0000","last_undegraded":"2024-01-19T04:56:25.947135+0000","last_fullsized":"2024-01-19T04:56:25.947135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:30:56.438301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.941883+0000","last_change":"2024-01-19T04:56:25.940964+0000","last_active":"2024-01-19T04:56:25.941883+0000","last_peered":"2024-01-19T04:56:25.941883+0000","last_clean":"2024-01-19T04:56:25.941883+0000","last_became_active":"2024-01-19T04:56:25.940589+0000","last_became_peered":"2024-01-19T04:56:25.940589+0000","last_unstale":"2024-01-19T04:56:25.941883+0000","last_undegraded":"2024-01-19T04:56:25.941883+0000","last_fullsized":"2024-01-19T04:56:25.941883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:17.691346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.951984+0000","last_change":"2024-01-19T04:56:24.925660+0000","last_active":"2024-01-19T04:56:25.951984+0000","last_peered":"2024-01-19T04:56:25.951984+0000","last_clean":"2024-01-19T04:56:25.951984+0000","last_became_active":"2024-01-19T04:56:24.925394+0000","last_became_peered":"2024-01-19T04:56:24.925394+0000","last_unstale":"2024-01-19T04:56:25.951984+0000","last_undegraded":"2024-01-19T04:56:25.951984+0000","last_fullsized":"2024-01-19T04:56:25.951984+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:48:32.505270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943198+0000","last_change":"2024-01-19T04:56:25.936910+0000","last_active":"2024-01-19T04:56:25.943198+0000","last_peered":"2024-01-19T04:56:25.943198+0000","last_clean":"2024-01-19T04:56:25.943198+0000","last_became_active":"2024-01-19T04:56:25.936658+0000","last_became_peered":"2024-01-19T04:56:25.936658+0000","last_unstale":"2024-01-19T04:56:25.943198+0000","last_undegraded":"2024-01-19T04:56:25.943198+0000","last_fullsized":"2024-01-19T04:56:25.943198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:38.155889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954265+0000","last_change":"2024-01-19T04:56:24.925809+0000","last_active":"2024-01-19T04:56:25.954265+0000","last_peered":"2024-01-19T04:56:25.954265+0000","last_clean":"2024-01-19T04:56:25.954265+0000","last_became_active":"2024-01-19T04:56:24.925436+0000","last_became_peered":"2024-01-19T04:56:24.925436+0000","last_unstale":"2024-01-19T04:56:25.954265+0000","last_undegraded":"2024-01-19T04:56:25.954265+0000","last_fullsized":"2024-01-19T04:56:25.954265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:04:44.398370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.941260+0000","last_change":"2024-01-19T04:56:25.939555+0000","last_active":"2024-01-19T04:56:25.941260+0000","last_peered":"2024-01-19T04:56:25.941260+0000","last_clean":"2024-01-19T04:56:25.941260+0000","last_became_active":"2024-01-19T04:56:25.939270+0000","last_became_peered":"2024-01-19T04:56:25.939270+0000","last_unstale":"2024-01-19T04:56:25.941260+0000","last_undegraded":"2024-01-19T04:56:25.941260+0000","last_fullsized":"2024-01-19T04:56:25.941260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:33:47.884613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926126+0000","last_change":"2024-01-19T04:56:24.923195+0000","last_active":"2024-01-19T04:56:25.926126+0000","last_peered":"2024-01-19T04:56:25.926126+0000","last_clean":"2024-01-19T04:56:25.926126+0000","last_became_active":"2024-01-19T04:56:24.922941+0000","last_became_peered":"2024-01-19T04:56:24.922941+0000","last_unstale":"2024-01-19T04:56:25.926126+0000","last_undegraded":"2024-01-19T04:56:25.926126+0000","last_fullsized":"2024-01-19T04:56:25.926126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:53.631079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943111+0000","last_change":"2024-01-19T04:56:25.934541+0000","last_active":"2024-01-19T04:56:25.943111+0000","last_peered":"2024-01-19T04:56:25.943111+0000","last_clean":"2024-01-19T04:56:25.943111+0000","last_became_active":"2024-01-19T04:56:25.934277+0000","last_became_peered":"2024-01-19T04:56:25.934277+0000","last_unstale":"2024-01-19T04:56:25.943111+0000","last_undegraded":"2024-01-19T04:56:25.943111+0000","last_fullsized":"2024-01-19T04:56:25.943111+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:59:03.271823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.925715+0000","last_change":"2024-01-19T04:56:24.924504+0000","last_active":"2024-01-19T04:56:25.925715+0000","last_peered":"2024-01-19T04:56:25.925715+0000","last_clean":"2024-01-19T04:56:25.925715+0000","last_became_active":"2024-01-19T04:56:24.924223+0000","last_became_peered":"2024-01-19T04:56:24.924223+0000","last_unstale":"2024-01-19T04:56:25.925715+0000","last_undegraded":"2024-01-19T04:56:25.925715+0000","last_fullsized":"2024-01-19T04:56:25.925715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:18:41.438563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950847+0000","last_change":"2024-01-19T04:56:25.945986+0000","last_active":"2024-01-19T04:56:25.950847+0000","last_peered":"2024-01-19T04:56:25.950847+0000","last_clean":"2024-01-19T04:56:25.950847+0000","last_became_active":"2024-01-19T04:56:25.945742+0000","last_became_peered":"2024-01-19T04:56:25.945742+0000","last_unstale":"2024-01-19T04:56:25.950847+0000","last_undegraded":"2024-01-19T04:56:25.950847+0000","last_fullsized":"2024-01-19T04:56:25.950847+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:12:35.815320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.927045+0000","last_change":"2024-01-19T04:56:24.928679+0000","last_active":"2024-01-19T04:56:25.927045+0000","last_peered":"2024-01-19T04:56:25.927045+0000","last_clean":"2024-01-19T04:56:25.927045+0000","last_became_active":"2024-01-19T04:56:24.928405+0000","last_became_peered":"2024-01-19T04:56:24.928405+0000","last_unstale":"2024-01-19T04:56:25.927045+0000","last_undegraded":"2024-01-19T04:56:25.927045+0000","last_fullsized":"2024-01-19T04:56:25.927045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:23.201603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954095+0000","last_change":"2024-01-19T04:56:25.948766+0000","last_active":"2024-01-19T04:56:25.954095+0000","last_peered":"2024-01-19T04:56:25.954095+0000","last_clean":"2024-01-19T04:56:25.954095+0000","last_became_active":"2024-01-19T04:56:25.948508+0000","last_became_peered":"2024-01-19T04:56:25.948508+0000","last_unstale":"2024-01-19T04:56:25.954095+0000","last_undegraded":"2024-01-19T04:56:25.954095+0000","last_fullsized":"2024-01-19T04:56:25.954095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:12:17.004437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.923542+0000","last_change":"2024-01-19T04:56:24.925942+0000","last_active":"2024-01-19T04:56:25.923542+0000","last_peered":"2024-01-19T04:56:25.923542+0000","last_clean":"2024-01-19T04:56:25.923542+0000","last_became_active":"2024-01-19T04:56:24.925656+0000","last_became_peered":"2024-01-19T04:56:24.925656+0000","last_unstale":"2024-01-19T04:56:25.923542+0000","last_undegraded":"2024-01-19T04:56:25.923542+0000","last_fullsized":"2024-01-19T04:56:25.923542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:32.417303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944856+0000","last_change":"2024-01-19T04:56:25.933886+0000","last_active":"2024-01-19T04:56:25.944856+0000","last_peered":"2024-01-19T04:56:25.944856+0000","last_clean":"2024-01-19T04:56:25.944856+0000","last_became_active":"2024-01-19T04:56:25.933586+0000","last_became_peered":"2024-01-19T04:56:25.933586+0000","last_unstale":"2024-01-19T04:56:25.944856+0000","last_undegraded":"2024-01-19T04:56:25.944856+0000","last_fullsized":"2024-01-19T04:56:25.944856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:58:19.400385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950999+0000","last_change":"2024-01-19T04:56:24.930228+0000","last_active":"2024-01-19T04:56:25.950999+0000","last_peered":"2024-01-19T04:56:25.950999+0000","last_clean":"2024-01-19T04:56:25.950999+0000","last_became_active":"2024-01-19T04:56:24.929966+0000","last_became_peered":"2024-01-19T04:56:24.929966+0000","last_unstale":"2024-01-19T04:56:25.950999+0000","last_undegraded":"2024-01-19T04:56:25.950999+0000","last_fullsized":"2024-01-19T04:56:25.950999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:03.086283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950825+0000","last_change":"2024-01-19T04:56:25.945682+0000","last_active":"2024-01-19T04:56:25.950825+0000","last_peered":"2024-01-19T04:56:25.950825+0000","last_clean":"2024-01-19T04:56:25.950825+0000","last_became_active":"2024-01-19T04:56:25.945308+0000","last_became_peered":"2024-01-19T04:56:25.945308+0000","last_unstale":"2024-01-19T04:56:25.950825+0000","last_undegraded":"2024-01-19T04:56:25.950825+0000","last_fullsized":"2024-01-19T04:56:25.950825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:43:41.276962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.959410+0000","last_change":"2024-01-19T04:56:24.932470+0000","last_active":"2024-01-19T04:56:25.959410+0000","last_peered":"2024-01-19T04:56:25.959410+0000","last_clean":"2024-01-19T04:56:25.959410+0000","last_became_active":"2024-01-19T04:56:24.930719+0000","last_became_peered":"2024-01-19T04:56:24.930719+0000","last_unstale":"2024-01-19T04:56:25.959410+0000","last_undegraded":"2024-01-19T04:56:25.959410+0000","last_fullsized":"2024-01-19T04:56:25.959410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:59:26.167612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945347+0000","last_change":"2024-01-19T04:56:25.943515+0000","last_active":"2024-01-19T04:56:25.945347+0000","last_peered":"2024-01-19T04:56:25.945347+0000","last_clean":"2024-01-19T04:56:25.945347+0000","last_became_active":"2024-01-19T04:56:25.943243+0000","last_became_peered":"2024-01-19T04:56:25.943243+0000","last_unstale":"2024-01-19T04:56:25.945347+0000","last_undegraded":"2024-01-19T04:56:25.945347+0000","last_fullsized":"2024-01-19T04:56:25.945347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:38.155889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950160+0000","last_change":"2024-01-19T04:56:24.926841+0000","last_active":"2024-01-19T04:56:25.950160+0000","last_peered":"2024-01-19T04:56:25.950160+0000","last_clean":"2024-01-19T04:56:25.950160+0000","last_became_active":"2024-01-19T04:56:24.926494+0000","last_became_peered":"2024-01-19T04:56:24.926494+0000","last_unstale":"2024-01-19T04:56:25.950160+0000","last_undegraded":"2024-01-19T04:56:25.950160+0000","last_fullsized":"2024-01-19T04:56:25.950160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:17:38.834388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943210+0000","last_change":"2024-01-19T04:56:25.935128+0000","last_active":"2024-01-19T04:56:25.943210+0000","last_peered":"2024-01-19T04:56:25.943210+0000","last_clean":"2024-01-19T04:56:25.943210+0000","last_became_active":"2024-01-19T04:56:25.934889+0000","last_became_peered":"2024-01-19T04:56:25.934889+0000","last_unstale":"2024-01-19T04:56:25.943210+0000","last_undegraded":"2024-01-19T04:56:25.943210+0000","last_fullsized":"2024-01-19T04:56:25.943210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:43:41.276962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.929425+0000","last_change":"2024-01-19T04:56:24.928217+0000","last_active":"2024-01-19T04:56:25.929425+0000","last_peered":"2024-01-19T04:56:25.929425+0000","last_clean":"2024-01-19T04:56:25.929425+0000","last_became_active":"2024-01-19T04:56:24.927573+0000","last_became_peered":"2024-01-19T04:56:24.927573+0000","last_unstale":"2024-01-19T04:56:25.929425+0000","last_undegraded":"2024-01-19T04:56:25.929425+0000","last_fullsized":"2024-01-19T04:56:25.929425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:43:00.441468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944954+0000","last_change":"2024-01-19T04:56:25.943118+0000","last_active":"2024-01-19T04:56:25.944954+0000","last_peered":"2024-01-19T04:56:25.944954+0000","last_clean":"2024-01-19T04:56:25.944954+0000","last_became_active":"2024-01-19T04:56:25.942634+0000","last_became_peered":"2024-01-19T04:56:25.942634+0000","last_unstale":"2024-01-19T04:56:25.944954+0000","last_undegraded":"2024-01-19T04:56:25.944954+0000","last_fullsized":"2024-01-19T04:56:25.944954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:02.466345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926277+0000","last_change":"2024-01-19T04:56:24.932474+0000","last_active":"2024-01-19T04:56:25.926277+0000","last_peered":"2024-01-19T04:56:25.926277+0000","last_clean":"2024-01-19T04:56:25.926277+0000","last_became_active":"2024-01-19T04:56:24.932247+0000","last_became_peered":"2024-01-19T04:56:24.932247+0000","last_unstale":"2024-01-19T04:56:25.926277+0000","last_undegraded":"2024-01-19T04:56:25.926277+0000","last_fullsized":"2024-01-19T04:56:25.926277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:01.460826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944116+0000","last_change":"2024-01-19T04:56:25.942182+0000","last_active":"2024-01-19T04:56:25.944116+0000","last_peered":"2024-01-19T04:56:25.944116+0000","last_clean":"2024-01-19T04:56:25.944116+0000","last_became_active":"2024-01-19T04:56:25.941922+0000","last_became_peered":"2024-01-19T04:56:25.941922+0000","last_unstale":"2024-01-19T04:56:25.944116+0000","last_undegraded":"2024-01-19T04:56:25.944116+0000","last_fullsized":"2024-01-19T04:56:25.944116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:02.088796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954953+0000","last_change":"2024-01-19T04:56:24.926332+0000","last_active":"2024-01-19T04:56:25.954953+0000","last_peered":"2024-01-19T04:56:25.954953+0000","last_clean":"2024-01-19T04:56:25.954953+0000","last_became_active":"2024-01-19T04:56:24.926092+0000","last_became_peered":"2024-01-19T04:56:24.926092+0000","last_unstale":"2024-01-19T04:56:25.954953+0000","last_undegraded":"2024-01-19T04:56:25.954953+0000","last_fullsized":"2024-01-19T04:56:25.954953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:43:37.150370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.937746+0000","last_change":"2024-01-19T04:56:25.932261+0000","last_active":"2024-01-19T04:56:25.937746+0000","last_peered":"2024-01-19T04:56:25.937746+0000","last_clean":"2024-01-19T04:56:25.937746+0000","last_became_active":"2024-01-19T04:56:25.931981+0000","last_became_peered":"2024-01-19T04:56:25.931981+0000","last_unstale":"2024-01-19T04:56:25.937746+0000","last_undegraded":"2024-01-19T04:56:25.937746+0000","last_fullsized":"2024-01-19T04:56:25.937746+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:59:01.049547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.954394+0000","last_change":"2024-01-19T04:56:24.928028+0000","last_active":"2024-01-19T04:56:25.954394+0000","last_peered":"2024-01-19T04:56:25.954394+0000","last_clean":"2024-01-19T04:56:25.954394+0000","last_became_active":"2024-01-19T04:56:24.927732+0000","last_became_peered":"2024-01-19T04:56:24.927732+0000","last_unstale":"2024-01-19T04:56:25.954394+0000","last_undegraded":"2024-01-19T04:56:25.954394+0000","last_fullsized":"2024-01-19T04:56:25.954394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:01.083277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.937120+0000","last_change":"2024-01-19T04:56:25.935252+0000","last_active":"2024-01-19T04:56:25.937120+0000","last_peered":"2024-01-19T04:56:25.937120+0000","last_clean":"2024-01-19T04:56:25.937120+0000","last_became_active":"2024-01-19T04:56:25.934762+0000","last_became_peered":"2024-01-19T04:56:25.934762+0000","last_unstale":"2024-01-19T04:56:25.937120+0000","last_undegraded":"2024-01-19T04:56:25.937120+0000","last_fullsized":"2024-01-19T04:56:25.937120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:49:02.466345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947994+0000","last_change":"2024-01-19T04:56:24.926522+0000","last_active":"2024-01-19T04:56:25.947994+0000","last_peered":"2024-01-19T04:56:25.947994+0000","last_clean":"2024-01-19T04:56:25.947994+0000","last_became_active":"2024-01-19T04:56:24.926234+0000","last_became_peered":"2024-01-19T04:56:24.926234+0000","last_unstale":"2024-01-19T04:56:25.947994+0000","last_undegraded":"2024-01-19T04:56:25.947994+0000","last_fullsized":"2024-01-19T04:56:25.947994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:51:56.602187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.943825+0000","last_change":"2024-01-19T04:56:25.940677+0000","last_active":"2024-01-19T04:56:25.943825+0000","last_peered":"2024-01-19T04:56:25.943825+0000","last_clean":"2024-01-19T04:56:25.943825+0000","last_became_active":"2024-01-19T04:56:25.940078+0000","last_became_peered":"2024-01-19T04:56:25.940078+0000","last_unstale":"2024-01-19T04:56:25.943825+0000","last_undegraded":"2024-01-19T04:56:25.943825+0000","last_fullsized":"2024-01-19T04:56:25.943825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:04.091802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947782+0000","last_change":"2024-01-19T04:56:24.925608+0000","last_active":"2024-01-19T04:56:25.947782+0000","last_peered":"2024-01-19T04:56:25.947782+0000","last_clean":"2024-01-19T04:56:25.947782+0000","last_became_active":"2024-01-19T04:56:24.925121+0000","last_became_peered":"2024-01-19T04:56:24.925121+0000","last_unstale":"2024-01-19T04:56:25.947782+0000","last_undegraded":"2024-01-19T04:56:25.947782+0000","last_fullsized":"2024-01-19T04:56:25.947782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:57:30.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.945044+0000","last_change":"2024-01-19T04:56:25.937633+0000","last_active":"2024-01-19T04:56:25.945044+0000","last_peered":"2024-01-19T04:56:25.945044+0000","last_clean":"2024-01-19T04:56:25.945044+0000","last_became_active":"2024-01-19T04:56:25.937362+0000","last_became_peered":"2024-01-19T04:56:25.937362+0000","last_unstale":"2024-01-19T04:56:25.945044+0000","last_undegraded":"2024-01-19T04:56:25.945044+0000","last_fullsized":"2024-01-19T04:56:25.945044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:50:10.958541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.925025+0000","last_change":"2024-01-19T04:56:24.928044+0000","last_active":"2024-01-19T04:56:25.925025+0000","last_peered":"2024-01-19T04:56:25.925025+0000","last_clean":"2024-01-19T04:56:25.925025+0000","last_became_active":"2024-01-19T04:56:24.927759+0000","last_became_peered":"2024-01-19T04:56:24.927759+0000","last_unstale":"2024-01-19T04:56:25.925025+0000","last_undegraded":"2024-01-19T04:56:25.925025+0000","last_fullsized":"2024-01-19T04:56:25.925025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:56:40.404698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947205+0000","last_change":"2024-01-19T04:56:25.943055+0000","last_active":"2024-01-19T04:56:25.947205+0000","last_peered":"2024-01-19T04:56:25.947205+0000","last_clean":"2024-01-19T04:56:25.947205+0000","last_became_active":"2024-01-19T04:56:25.942775+0000","last_became_peered":"2024-01-19T04:56:25.942775+0000","last_unstale":"2024-01-19T04:56:25.947205+0000","last_undegraded":"2024-01-19T04:56:25.947205+0000","last_fullsized":"2024-01-19T04:56:25.947205+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:56:41.410217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946232+0000","last_change":"2024-01-19T04:56:24.926337+0000","last_active":"2024-01-19T04:56:25.946232+0000","last_peered":"2024-01-19T04:56:25.946232+0000","last_clean":"2024-01-19T04:56:25.946232+0000","last_became_active":"2024-01-19T04:56:24.926083+0000","last_became_peered":"2024-01-19T04:56:24.926083+0000","last_unstale":"2024-01-19T04:56:25.946232+0000","last_undegraded":"2024-01-19T04:56:25.946232+0000","last_fullsized":"2024-01-19T04:56:25.946232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:13.511105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.940590+0000","last_change":"2024-01-19T04:56:25.934203+0000","last_active":"2024-01-19T04:56:25.940590+0000","last_peered":"2024-01-19T04:56:25.940590+0000","last_clean":"2024-01-19T04:56:25.940590+0000","last_became_active":"2024-01-19T04:56:25.933933+0000","last_became_peered":"2024-01-19T04:56:25.933933+0000","last_unstale":"2024-01-19T04:56:25.940590+0000","last_undegraded":"2024-01-19T04:56:25.940590+0000","last_fullsized":"2024-01-19T04:56:25.940590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:11.475502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.936853+0000","last_change":"2024-01-19T04:56:24.930127+0000","last_active":"2024-01-19T04:56:25.936853+0000","last_peered":"2024-01-19T04:56:25.936853+0000","last_clean":"2024-01-19T04:56:25.936853+0000","last_became_active":"2024-01-19T04:56:24.929877+0000","last_became_peered":"2024-01-19T04:56:24.929877+0000","last_unstale":"2024-01-19T04:56:25.936853+0000","last_undegraded":"2024-01-19T04:56:25.936853+0000","last_fullsized":"2024-01-19T04:56:25.936853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:13:56.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947294+0000","last_change":"2024-01-19T04:56:25.941008+0000","last_active":"2024-01-19T04:56:25.947294+0000","last_peered":"2024-01-19T04:56:25.947294+0000","last_clean":"2024-01-19T04:56:25.947294+0000","last_became_active":"2024-01-19T04:56:25.940703+0000","last_became_peered":"2024-01-19T04:56:25.940703+0000","last_unstale":"2024-01-19T04:56:25.947294+0000","last_undegraded":"2024-01-19T04:56:25.947294+0000","last_fullsized":"2024-01-19T04:56:25.947294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:31:36.515465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926094+0000","last_change":"2024-01-19T04:56:24.929045+0000","last_active":"2024-01-19T04:56:25.926094+0000","last_peered":"2024-01-19T04:56:25.926094+0000","last_clean":"2024-01-19T04:56:25.926094+0000","last_became_active":"2024-01-19T04:56:24.928761+0000","last_became_peered":"2024-01-19T04:56:24.928761+0000","last_unstale":"2024-01-19T04:56:25.926094+0000","last_undegraded":"2024-01-19T04:56:25.926094+0000","last_fullsized":"2024-01-19T04:56:25.926094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:42:52.025292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950677+0000","last_change":"2024-01-19T04:56:25.946061+0000","last_active":"2024-01-19T04:56:25.950677+0000","last_peered":"2024-01-19T04:56:25.950677+0000","last_clean":"2024-01-19T04:56:25.950677+0000","last_became_active":"2024-01-19T04:56:25.945574+0000","last_became_peered":"2024-01-19T04:56:25.945574+0000","last_unstale":"2024-01-19T04:56:25.950677+0000","last_undegraded":"2024-01-19T04:56:25.950677+0000","last_fullsized":"2024-01-19T04:56:25.950677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:58:19.400385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.924698+0000","last_change":"2024-01-19T04:56:24.930342+0000","last_active":"2024-01-19T04:56:25.924698+0000","last_peered":"2024-01-19T04:56:25.924698+0000","last_clean":"2024-01-19T04:56:25.924698+0000","last_became_active":"2024-01-19T04:56:24.929993+0000","last_became_peered":"2024-01-19T04:56:24.929993+0000","last_unstale":"2024-01-19T04:56:25.924698+0000","last_undegraded":"2024-01-19T04:56:25.924698+0000","last_fullsized":"2024-01-19T04:56:25.924698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:33:16.685827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.946938+0000","last_change":"2024-01-19T04:56:25.938198+0000","last_active":"2024-01-19T04:56:25.946938+0000","last_peered":"2024-01-19T04:56:25.946938+0000","last_clean":"2024-01-19T04:56:25.946938+0000","last_became_active":"2024-01-19T04:56:25.937953+0000","last_became_peered":"2024-01-19T04:56:25.937953+0000","last_unstale":"2024-01-19T04:56:25.946938+0000","last_undegraded":"2024-01-19T04:56:25.946938+0000","last_fullsized":"2024-01-19T04:56:25.946938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:55:17.134541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953468+0000","last_change":"2024-01-19T04:56:24.928922+0000","last_active":"2024-01-19T04:56:25.953468+0000","last_peered":"2024-01-19T04:56:25.953468+0000","last_clean":"2024-01-19T04:56:25.953468+0000","last_became_active":"2024-01-19T04:56:24.928428+0000","last_became_peered":"2024-01-19T04:56:24.928428+0000","last_unstale":"2024-01-19T04:56:25.953468+0000","last_undegraded":"2024-01-19T04:56:25.953468+0000","last_fullsized":"2024-01-19T04:56:25.953468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:56:10.280240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.950784+0000","last_change":"2024-01-19T04:56:25.945424+0000","last_active":"2024-01-19T04:56:25.950784+0000","last_peered":"2024-01-19T04:56:25.950784+0000","last_clean":"2024-01-19T04:56:25.950784+0000","last_became_active":"2024-01-19T04:56:25.945145+0000","last_became_peered":"2024-01-19T04:56:25.945145+0000","last_unstale":"2024-01-19T04:56:25.950784+0000","last_undegraded":"2024-01-19T04:56:25.950784+0000","last_fullsized":"2024-01-19T04:56:25.950784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:24.207122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.923672+0000","last_change":"2024-01-19T04:56:24.929929+0000","last_active":"2024-01-19T04:56:25.923672+0000","last_peered":"2024-01-19T04:56:25.923672+0000","last_clean":"2024-01-19T04:56:25.923672+0000","last_became_active":"2024-01-19T04:56:24.929635+0000","last_became_peered":"2024-01-19T04:56:24.929635+0000","last_unstale":"2024-01-19T04:56:25.923672+0000","last_undegraded":"2024-01-19T04:56:25.923672+0000","last_fullsized":"2024-01-19T04:56:25.923672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:43:00.441468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947580+0000","last_change":"2024-01-19T04:56:25.944802+0000","last_active":"2024-01-19T04:56:25.947580+0000","last_peered":"2024-01-19T04:56:25.947580+0000","last_clean":"2024-01-19T04:56:25.947580+0000","last_became_active":"2024-01-19T04:56:25.944519+0000","last_became_peered":"2024-01-19T04:56:25.944519+0000","last_unstale":"2024-01-19T04:56:25.947580+0000","last_undegraded":"2024-01-19T04:56:25.947580+0000","last_fullsized":"2024-01-19T04:56:25.947580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:24:54.636598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.928037+0000","last_change":"2024-01-19T04:56:24.928531+0000","last_active":"2024-01-19T04:56:25.928037+0000","last_peered":"2024-01-19T04:56:25.928037+0000","last_clean":"2024-01-19T04:56:25.928037+0000","last_became_active":"2024-01-19T04:56:24.928185+0000","last_became_peered":"2024-01-19T04:56:24.928185+0000","last_unstale":"2024-01-19T04:56:25.928037+0000","last_undegraded":"2024-01-19T04:56:25.928037+0000","last_fullsized":"2024-01-19T04:56:25.928037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:32.417303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947528+0000","last_change":"2024-01-19T04:56:25.943570+0000","last_active":"2024-01-19T04:56:25.947528+0000","last_peered":"2024-01-19T04:56:25.947528+0000","last_clean":"2024-01-19T04:56:25.947528+0000","last_became_active":"2024-01-19T04:56:25.943291+0000","last_became_peered":"2024-01-19T04:56:25.943291+0000","last_unstale":"2024-01-19T04:56:25.947528+0000","last_undegraded":"2024-01-19T04:56:25.947528+0000","last_fullsized":"2024-01-19T04:56:25.947528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:26:33.422822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.923319+0000","last_change":"2024-01-19T04:56:24.920947+0000","last_active":"2024-01-19T04:56:25.923319+0000","last_peered":"2024-01-19T04:56:25.923319+0000","last_clean":"2024-01-19T04:56:25.923319+0000","last_became_active":"2024-01-19T04:56:24.920655+0000","last_became_peered":"2024-01-19T04:56:24.920655+0000","last_unstale":"2024-01-19T04:56:25.923319+0000","last_undegraded":"2024-01-19T04:56:25.923319+0000","last_fullsized":"2024-01-19T04:56:25.923319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:30:01.083277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.942087+0000","last_change":"2024-01-19T04:56:25.940355+0000","last_active":"2024-01-19T04:56:25.942087+0000","last_peered":"2024-01-19T04:56:25.942087+0000","last_clean":"2024-01-19T04:56:25.942087+0000","last_became_active":"2024-01-19T04:56:25.940092+0000","last_became_peered":"2024-01-19T04:56:25.940092+0000","last_unstale":"2024-01-19T04:56:25.942087+0000","last_undegraded":"2024-01-19T04:56:25.942087+0000","last_fullsized":"2024-01-19T04:56:25.942087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:37:04.091802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.927530+0000","last_change":"2024-01-19T04:56:24.927768+0000","last_active":"2024-01-19T04:56:25.927530+0000","last_peered":"2024-01-19T04:56:25.927530+0000","last_clean":"2024-01-19T04:56:25.927530+0000","last_became_active":"2024-01-19T04:56:24.927215+0000","last_became_peered":"2024-01-19T04:56:24.927215+0000","last_unstale":"2024-01-19T04:56:25.927530+0000","last_undegraded":"2024-01-19T04:56:25.927530+0000","last_fullsized":"2024-01-19T04:56:25.927530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:47:13.511105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.947850+0000","last_change":"2024-01-19T04:56:25.946278+0000","last_active":"2024-01-19T04:56:25.947850+0000","last_peered":"2024-01-19T04:56:25.947850+0000","last_clean":"2024-01-19T04:56:25.947850+0000","last_became_active":"2024-01-19T04:56:25.946014+0000","last_became_peered":"2024-01-19T04:56:25.946014+0000","last_unstale":"2024-01-19T04:56:25.947850+0000","last_undegraded":"2024-01-19T04:56:25.947850+0000","last_fullsized":"2024-01-19T04:56:25.947850+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:25:48.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926232+0000","last_change":"2024-01-19T04:56:24.929537+0000","last_active":"2024-01-19T04:56:25.926232+0000","last_peered":"2024-01-19T04:56:25.926232+0000","last_clean":"2024-01-19T04:56:25.926232+0000","last_became_active":"2024-01-19T04:56:24.929276+0000","last_became_peered":"2024-01-19T04:56:24.929276+0000","last_unstale":"2024-01-19T04:56:25.926232+0000","last_undegraded":"2024-01-19T04:56:25.926232+0000","last_fullsized":"2024-01-19T04:56:25.926232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:07:53.445003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.944580+0000","last_change":"2024-01-19T04:56:25.941715+0000","last_active":"2024-01-19T04:56:25.944580+0000","last_peered":"2024-01-19T04:56:25.944580+0000","last_clean":"2024-01-19T04:56:25.944580+0000","last_became_active":"2024-01-19T04:56:25.941460+0000","last_became_peered":"2024-01-19T04:56:25.941460+0000","last_unstale":"2024-01-19T04:56:25.944580+0000","last_undegraded":"2024-01-19T04:56:25.944580+0000","last_fullsized":"2024-01-19T04:56:25.944580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:27:22.525627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.926873+0000","last_change":"2024-01-19T04:56:24.929140+0000","last_active":"2024-01-19T04:56:25.926873+0000","last_peered":"2024-01-19T04:56:25.926873+0000","last_clean":"2024-01-19T04:56:25.926873+0000","last_became_active":"2024-01-19T04:56:24.928854+0000","last_became_peered":"2024-01-19T04:56:24.928854+0000","last_unstale":"2024-01-19T04:56:25.926873+0000","last_undegraded":"2024-01-19T04:56:25.926873+0000","last_fullsized":"2024-01-19T04:56:25.926873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:39:48.029021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.953257+0000","last_change":"2024-01-19T04:56:25.949297+0000","last_active":"2024-01-19T04:56:25.953257+0000","last_peered":"2024-01-19T04:56:25.953257+0000","last_clean":"2024-01-19T04:56:25.953257+0000","last_became_active":"2024-01-19T04:56:25.949004+0000","last_became_peered":"2024-01-19T04:56:25.949004+0000","last_unstale":"2024-01-19T04:56:25.953257+0000","last_undegraded":"2024-01-19T04:56:25.953257+0000","last_fullsized":"2024-01-19T04:56:25.953257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:24.907287+0000","last_clean_scrub_stamp":"2024-01-19T04:56:24.907287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:43:41.276962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-19T04:56:25.937346+0000","last_change":"2024-01-19T04:56:24.927874+0000","last_active":"2024-01-19T04:56:25.937346+0000","last_peered":"2024-01-19T04:56:25.937346+0000","last_clean":"2024-01-19T04:56:25.937346+0000","last_became_active":"2024-01-19T04:56:24.927592+0000","last_became_peered":"2024-01-19T04:56:24.927592+0000","last_unstale":"2024-01-19T04:56:25.937346+0000","last_undegraded":"2024-01-19T04:56:25.937346+0000","last_fullsized":"2024-01-19T04:56:25.937346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:56:23.901768+0000","last_clean_scrub_stamp":"2024-01-19T04:56:23.901768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:27:21.520108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-19T04:56:24.924463+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:56:24.924463+0000","last_peered":"2024-01-19T04:56:24.924463+0000","last_clean":"2024-01-19T04:56:24.924463+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:56:24.924463+0000","last_undegraded":"2024-01-19T04:56:24.924463+0000","last_fullsized":"2024-01-19T04:56:24.924463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":736065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":4.7800000000000002,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.629},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.79400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.439},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.36599999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.41399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63}]},{"osd":3,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.314},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.437}]},{"osd":7,"last update":"Fri Jan 19 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.504}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.373}]},{"osd":3,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.39700000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.626},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.65300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.251},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.23899999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.27000000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.624},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":735975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.223},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.379}]},{"osd":2,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.252},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.505},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Jan 19 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:56:29.279 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-19T04:56:29.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-19T04:56:29.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.421+0000 7fbe4814c640 1 Processor -- start 2024-01-19T04:56:29.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.421+0000 7fbe4814c640 1 -- start start 2024-01-19T04:56:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe4814c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe400ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe4814c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbe4015da00 0x7fbe400b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe4814c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe40066690 con 0x7fbe4015af30 2024-01-19T04:56:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe4814c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe400ace00 con 0x7fbe4015da00 2024-01-19T04:56:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe4814c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4015df70 con 0x7fbe4006bcd0 2024-01-19T04:56:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe45ee8640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbe4015af30 0x7fbe4006b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39644/0 (socket says 172.21.15.31:39644) 2024-01-19T04:56:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe45ee8640 1 -- 172.21.15.31:0/2064581292 learned_addr learned my addr 172.21.15.31:0/2064581292 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe466e9640 1 --2- 172.21.15.31:0/2064581292 >> v2:172.21.15.31:3300/0 conn(0x7fbe4015da00 0x7fbe400b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.422+0000 7fbe456e7640 1 --2- 172.21.15.31:0/2064581292 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe400ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe466e9640 1 -- 172.21.15.31:0/2064581292 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 msgr2=0x7fbe400ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe466e9640 1 --2- 172.21.15.31:0/2064581292 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe400ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe466e9640 1 -- 172.21.15.31:0/2064581292 >> v1:172.21.15.31:6789/0 conn(0x7fbe4015af30 legacy=0x7fbe4006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe466e9640 1 -- 172.21.15.31:0/2064581292 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4015e250 con 0x7fbe4015da00 2024-01-19T04:56:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe44ee6640 1 -- 172.21.15.31:0/2064581292 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109210480 0 0) 0x7fbe40066690 con 0x7fbe4015af30 2024-01-19T04:56:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe466e9640 1 --2- 172.21.15.31:0/2064581292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400b2b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009d90 tx=0x7fbe3002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba1cef8e9afef08 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe44ee6640 1 -- 172.21.15.31:0/2064581292 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3002cc50 con 0x7fbe4015da00 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.423+0000 7fbe44ee6640 1 -- 172.21.15.31:0/2064581292 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3002cdb0 con 0x7fbe4015da00 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe44ee6640 1 -- 172.21.15.31:0/2064581292 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30034660 con 0x7fbe4015da00 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 -- 172.21.15.31:0/2064581292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 msgr2=0x7fbe400b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 --2- 172.21.15.31:0/2064581292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400b2b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009d90 tx=0x7fbe3002bea0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 -- 172.21.15.31:0/2064581292 shutdown_connections 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 --2- 172.21.15.31:0/2064581292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400b2b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 --2- 172.21.15.31:0/2064581292 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe400ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 -- 172.21.15.31:0/2064581292 >> 172.21.15.31:0/2064581292 conn(0x7fbe40070bf0 msgr2=0x7fbe40071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 -- 172.21.15.31:0/2064581292 shutdown_connections 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.424+0000 7fbe4814c640 1 -- 172.21.15.31:0/2064581292 wait complete. 2024-01-19T04:56:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.425+0000 7fbe4814c640 1 Processor -- start 2024-01-19T04:56:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.425+0000 7fbe4814c640 1 -- start start 2024-01-19T04:56:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe4814c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe401550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe4814c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe4015af30 0x7fbe4015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe4814c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe4814c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400cddc0 con 0x7fbe4015da00 2024-01-19T04:56:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe4814c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40155610 con 0x7fbe4015af30 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe4814c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400cd1d0 con 0x7fbe4006bcd0 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58284/0 (socket says 172.21.15.31:58284) 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 -- 172.21.15.31:0/3926217155 learned_addr learned my addr 172.21.15.31:0/3926217155 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe45ee8640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe401550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 -- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 msgr2=0x7fbe401550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe456e7640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe4015af30 0x7fbe4015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe401550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 -- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe4015af30 msgr2=0x7fbe4015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe4015af30 0x7fbe4015fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe466e9640 1 -- 172.21.15.31:0/3926217155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30009a70 con 0x7fbe4015da00 2024-01-19T04:56:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe45ee8640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe401550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.426+0000 7fbe456e7640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe4015af30 0x7fbe4015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.427+0000 7fbe466e9640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400ce250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009d90 tx=0x7fbe30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.427+0000 7fbe3e7fc640 1 -- 172.21.15.31:0/3926217155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30003b90 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.427+0000 7fbe3e7fc640 1 -- 172.21.15.31:0/3926217155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30003d10 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.427+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4015b490 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.427+0000 7fbe3e7fc640 1 -- 172.21.15.31:0/3926217155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30004270 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.427+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe400cd770 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.428+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4006f640 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.428+0000 7fbe3e7fc640 1 -- 172.21.15.31:0/3926217155 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fbe300043d0 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.429+0000 7fbe3e7fc640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbe2c06aed0 0x7fbe2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.429+0000 7fbe3e7fc640 1 -- 172.21.15.31:0/3926217155 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe300abf00 con 0x7fbe4015da00 2024-01-19T04:56:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.429+0000 7fbe45ee8640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbe2c06aed0 0x7fbe2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.429+0000 7fbe45ee8640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbe2c06aed0 0x7fbe2c06d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000f60 tx=0x7fbe38009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.433+0000 7fbe3e7fc640 1 -- 172.21.15.31:0/3926217155 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbe30076e60 con 0x7fbe4015da00 2024-01-19T04:56:29.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.683+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbe40074900 con 0x7fbe4015da00 2024-01-19T04:56:29.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.686+0000 7fbe3e7fc640 1 -- 172.21.15.31:0/3926217155 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbe3007a480 con 0x7fbe4015da00 2024-01-19T04:56:29.685 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:56:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:56:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2024-01-19T04:56:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:56:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-19T04:56:29.686 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:56:29.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbe2c06aed0 msgr2=0x7fbe2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbe2c06aed0 0x7fbe2c06d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000f60 tx=0x7fbe38009040 comp rx=0 tx=0).stop 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 msgr2=0x7fbe400ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400ce250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009d90 tx=0x7fbe30004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 shutdown_connections 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbe2c06aed0 0x7fbe2c06d390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe4015da00 0x7fbe400ce250 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe4015af30 0x7fbe4015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 --2- 172.21.15.31:0/3926217155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe4006bcd0 0x7fbe401550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 >> 172.21.15.31:0/3926217155 conn(0x7fbe40070bf0 msgr2=0x7fbe4006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:29.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 shutdown_connections 2024-01-19T04:56:29.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.688+0000 7fbe4814c640 1 -- 172.21.15.31:0/3926217155 wait complete. 2024-01-19T04:56:29.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:29.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.799+0000 7fddd06d5640 1 Processor -- start 2024-01-19T04:56:29.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.799+0000 7fddd06d5640 1 -- start start 2024-01-19T04:56:29.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.800+0000 7fddd06d5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fddc8160bb0 0x7fddc80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.800+0000 7fddd06d5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80d1600 0x7fddc80d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.800+0000 7fddd06d5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc80d06b0 con 0x7fddc80cfd10 2024-01-19T04:56:29.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.800+0000 7fddd06d5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc80b66b0 con 0x7fddc8160bb0 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.800+0000 7fddd06d5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80d1b90 con 0x7fddc80d1600 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcec72640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80d1600 0x7fddc80d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddce471640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fddc8160bb0 0x7fddc80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcec72640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80d1600 0x7fddc80d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:39200/0 (socket says 172.21.15.31:39200) 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddce471640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fddc8160bb0 0x7fddc80b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:48022/0 (socket says 172.21.15.31:48022) 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcec72640 1 -- 172.21.15.31:0/4271776743 learned_addr learned my addr 172.21.15.31:0/4271776743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcdc70640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fddc80cfd10 0x7fddc80d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:45880/0 (socket says 172.21.15.31:45880) 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcec72640 1 -- 172.21.15.31:0/4271776743 >> v2:172.21.15.137:3300/0 conn(0x7fddc8160bb0 msgr2=0x7fddc80b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcec72640 1 --2- 172.21.15.31:0/4271776743 >> v2:172.21.15.137:3300/0 conn(0x7fddc8160bb0 0x7fddc80b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcec72640 1 -- 172.21.15.31:0/4271776743 >> v1:172.21.15.137:6789/0 conn(0x7fddc80cfd10 legacy=0x7fddc80d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddcec72640 1 -- 172.21.15.31:0/4271776743 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80d1e70 con 0x7fddc80d1600 2024-01-19T04:56:29.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.801+0000 7fddce471640 1 --2- 172.21.15.31:0/4271776743 >> v2:172.21.15.137:3300/0 conn(0x7fddc8160bb0 0x7fddc80b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:29.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.802+0000 7fddcec72640 1 --2- 172.21.15.31:0/4271776743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80d1600 0x7fddc80d0170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009e50 tx=0x7fddb802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3930c0287a6d032b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.802+0000 7fddcd46f640 1 -- 172.21.15.31:0/4271776743 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb802cb00 con 0x7fddc80d1600 2024-01-19T04:56:29.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.802+0000 7fddcd46f640 1 -- 172.21.15.31:0/4271776743 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb802cc60 con 0x7fddc80d1600 2024-01-19T04:56:29.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.802+0000 7fddcd46f640 1 -- 172.21.15.31:0/4271776743 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80343f0 con 0x7fddc80d1600 2024-01-19T04:56:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.802+0000 7fddd06d5640 1 -- 172.21.15.31:0/4271776743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80d1600 msgr2=0x7fddc80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.802+0000 7fddd06d5640 1 --2- 172.21.15.31:0/4271776743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80d1600 0x7fddc80d0170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009e50 tx=0x7fddb802bf60 comp rx=0 tx=0).stop 2024-01-19T04:56:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.803+0000 7fddd06d5640 1 -- 172.21.15.31:0/4271776743 shutdown_connections 2024-01-19T04:56:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.803+0000 7fddd06d5640 1 --2- 172.21.15.31:0/4271776743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80d1600 0x7fddc80d0170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.803+0000 7fddd06d5640 1 --2- 172.21.15.31:0/4271776743 >> v2:172.21.15.137:3300/0 conn(0x7fddc8160bb0 0x7fddc80b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.803+0000 7fddd06d5640 1 -- 172.21.15.31:0/4271776743 >> 172.21.15.31:0/4271776743 conn(0x7fddc809c600 msgr2=0x7fddc80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.803+0000 7fddd06d5640 1 -- 172.21.15.31:0/4271776743 shutdown_connections 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.803+0000 7fddd06d5640 1 -- 172.21.15.31:0/4271776743 wait complete. 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 Processor -- start 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 -- start start 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80cfd10 0x7fddc80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddc80d1600 0x7fddc8148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8160bb0 0x7fddc8149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc81f77f0 con 0x7fddc8160bb0 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc81f6a30 con 0x7fddc80d1600 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.804+0000 7fddd06d5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc81f6d10 con 0x7fddc80cfd10 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8160bb0 0x7fddc8149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8160bb0 0x7fddc8149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37026/0 (socket says 172.21.15.31:37026) 2024-01-19T04:56:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 -- 172.21.15.31:0/322293275 learned_addr learned my addr 172.21.15.31:0/322293275 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddce471640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80cfd10 0x7fddc80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcdc70640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddc80d1600 0x7fddc8148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 -- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80cfd10 msgr2=0x7fddc80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80cfd10 0x7fddc80ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 -- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddc80d1600 msgr2=0x7fddc8148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddc80d1600 0x7fddc8148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 -- 172.21.15.31:0/322293275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009400 con 0x7fddc8160bb0 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcdc70640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddc80d1600 0x7fddc8148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddcec72640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8160bb0 0x7fddc8149370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fddb8036040 tx=0x7fddb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddc6ffd640 1 -- 172.21.15.31:0/322293275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8009510 con 0x7fddc8160bb0 2024-01-19T04:56:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddc6ffd640 1 -- 172.21.15.31:0/322293275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb803e450 con 0x7fddc8160bb0 2024-01-19T04:56:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb8009b30 con 0x7fddc8160bb0 2024-01-19T04:56:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddc6ffd640 1 -- 172.21.15.31:0/322293275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb803e820 con 0x7fddc8160bb0 2024-01-19T04:56:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.805+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc81f72d0 con 0x7fddc8160bb0 2024-01-19T04:56:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.806+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc80696e0 con 0x7fddc8160bb0 2024-01-19T04:56:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.807+0000 7fddc6ffd640 1 -- 172.21.15.31:0/322293275 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fddb803e980 con 0x7fddc8160bb0 2024-01-19T04:56:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.807+0000 7fddc6ffd640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fddb406aed0 0x7fddb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.807+0000 7fddc6ffd640 1 -- 172.21.15.31:0/322293275 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb80ac360 con 0x7fddc8160bb0 2024-01-19T04:56:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.808+0000 7fddce471640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fddb406aed0 0x7fddb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.808+0000 7fddce471640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fddb406aed0 0x7fddb406d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009c60 tx=0x7fddc0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:29.812+0000 7fddc6ffd640 1 -- 172.21.15.31:0/322293275 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fddb8077220 con 0x7fddc8160bb0 2024-01-19T04:56:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.025+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddc80ca160 con 0x7fddc8160bb0 2024-01-19T04:56:30.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.026+0000 7fddc6ffd640 1 -- 172.21.15.31:0/322293275 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fddb807a840 con 0x7fddc8160bb0 2024-01-19T04:56:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:30.025 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:56:25.278275+0000","modified":"2024-01-19T04:56:27.276135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5259":{"gid":5259,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/968278604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":968278604},{"type":"v1","addr":"172.21.15.31:6835","nonce":968278604}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-19T04:56:30.026 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 14 2024-01-19T04:56:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fddb406aed0 msgr2=0x7fddb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fddb406aed0 0x7fddb406d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009c60 tx=0x7fddc0008040 comp rx=0 tx=0).stop 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8160bb0 msgr2=0x7fddc8149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8160bb0 0x7fddc8149370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fddb8036040 tx=0x7fddb8004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 shutdown_connections 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fddb406aed0 0x7fddb406d390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8160bb0 0x7fddc8149370 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddc80d1600 0x7fddc8148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 --2- 172.21.15.31:0/322293275 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddc80cfd10 0x7fddc80ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.027+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 >> 172.21.15.31:0/322293275 conn(0x7fddc809c600 msgr2=0x7fddc815c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.028+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 shutdown_connections 2024-01-19T04:56:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.028+0000 7fddd06d5640 1 -- 172.21.15.31:0/322293275 wait complete. 2024-01-19T04:56:30.440 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-19T04:56:25.278275+0000', 'modified': '2024-01-19T04:56:27.276135+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5259}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5259': {'gid': 5259, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/968278604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 968278604}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 968278604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-19T04:56:30.441 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-19T04:56:30.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-19T04:56:30.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.550+0000 7f3557df8640 1 Processor -- start 2024-01-19T04:56:30.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.550+0000 7f3557df8640 1 -- start start 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3557df8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35500d11b0 0x7f35500d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3557df8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35500d1fe0 0x7f35500b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3557df8640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35500cef10 con 0x7f355015d700 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3557df8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35500ce300 con 0x7f35500d11b0 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3557df8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35500ce5e0 con 0x7f35500d1fe0 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3555393640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35500d11b0 0x7f35500d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3555393640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35500d11b0 0x7f35500d3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37042/0 (socket says 172.21.15.31:37042) 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3555393640 1 -- 172.21.15.31:0/2336985512 learned_addr learned my addr 172.21.15.31:0/2336985512 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3556395640 1 --2- 172.21.15.31:0/2336985512 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35500d1fe0 0x7f35500b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3555393640 1 -- 172.21.15.31:0/2336985512 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35500d1fe0 msgr2=0x7f35500b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3555393640 1 --2- 172.21.15.31:0/2336985512 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35500d1fe0 0x7f35500b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3555393640 1 -- 172.21.15.31:0/2336985512 >> v1:172.21.15.137:6789/0 conn(0x7f355015d700 legacy=0x7f35500d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.551+0000 7f3555393640 1 -- 172.21.15.31:0/2336985512 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500ce8c0 con 0x7f35500d11b0 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.552+0000 7f3555393640 1 --2- 172.21.15.31:0/2336985512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d11b0 0x7f35500d3e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3540009870 tx=0x7f354002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1380c53d0827904a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.552+0000 7f3554b92640 1 -- 172.21.15.31:0/2336985512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354002f840 con 0x7f35500d11b0 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.552+0000 7f3554b92640 1 -- 172.21.15.31:0/2336985512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354002f9a0 con 0x7f35500d11b0 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.552+0000 7f3554b92640 1 -- 172.21.15.31:0/2336985512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354002fd30 con 0x7f35500d11b0 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 -- 172.21.15.31:0/2336985512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d11b0 msgr2=0x7f35500d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 --2- 172.21.15.31:0/2336985512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d11b0 0x7f35500d3e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3540009870 tx=0x7f354002ba20 comp rx=0 tx=0).stop 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 -- 172.21.15.31:0/2336985512 shutdown_connections 2024-01-19T04:56:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 --2- 172.21.15.31:0/2336985512 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35500d1fe0 0x7f35500b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 --2- 172.21.15.31:0/2336985512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d11b0 0x7f35500d3e10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 -- 172.21.15.31:0/2336985512 >> 172.21.15.31:0/2336985512 conn(0x7f35500d32f0 msgr2=0x7f35500d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 -- 172.21.15.31:0/2336985512 shutdown_connections 2024-01-19T04:56:30.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.553+0000 7f3557df8640 1 -- 172.21.15.31:0/2336985512 wait complete. 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.554+0000 7f3557df8640 1 Processor -- start 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.554+0000 7f3557df8640 1 -- start start 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3557df8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35500d11b0 0x7f35500c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3557df8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d1fe0 0x7f35500c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3557df8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f355015d700 0x7f3550150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3557df8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35500c8960 con 0x7f35500d1fe0 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3557df8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35500cbe30 con 0x7f35500d11b0 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3557df8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35500cc110 con 0x7f355015d700 2024-01-19T04:56:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d1fe0 0x7f35500c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d1fe0 0x7f35500c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37048/0 (socket says 172.21.15.31:37048) 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 -- 172.21.15.31:0/3110041977 learned_addr learned my addr 172.21.15.31:0/3110041977 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555b94640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35500d11b0 0x7f35500c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3556395640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f355015d700 0x7f3550150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 -- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f355015d700 msgr2=0x7f3550150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f355015d700 0x7f3550150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 -- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35500d11b0 msgr2=0x7f35500c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35500d11b0 0x7f35500c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555393640 1 -- 172.21.15.31:0/3110041977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35400094d0 con 0x7f35500d1fe0 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.555+0000 7f3555b94640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35500d11b0 0x7f35500c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.556+0000 7f3555393640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d1fe0 0x7f35500c9d20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3540000c00 tx=0x7f3540004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.556+0000 7f35467fc640 1 -- 172.21.15.31:0/3110041977 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540037c40 con 0x7f35500d1fe0 2024-01-19T04:56:30.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.556+0000 7f35467fc640 1 -- 172.21.15.31:0/3110041977 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3540037da0 con 0x7f35500d1fe0 2024-01-19T04:56:30.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.556+0000 7f35467fc640 1 -- 172.21.15.31:0/3110041977 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540041640 con 0x7f35500d1fe0 2024-01-19T04:56:30.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.556+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35500cc3f0 con 0x7f35500d1fe0 2024-01-19T04:56:30.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.556+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35500cc9b0 con 0x7f35500d1fe0 2024-01-19T04:56:30.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.558+0000 7f35467fc640 1 -- 172.21.15.31:0/3110041977 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f35400417a0 con 0x7f35500d1fe0 2024-01-19T04:56:30.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.558+0000 7f35467fc640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f353406af00 0x7f353406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.558+0000 7f35467fc640 1 -- 172.21.15.31:0/3110041977 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f354003c070 con 0x7f35500d1fe0 2024-01-19T04:56:30.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.558+0000 7f3555b94640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f353406af00 0x7f353406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.559+0000 7f355a9c4740 1 Processor -- start 2024-01-19T04:56:30.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.559+0000 7f3555b94640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f353406af00 0x7f353406d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3548001660 tx=0x7f3548009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.559+0000 7f355a9c4740 1 -- start start 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a6fe0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a7010020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a6fe0770 0x55d0a7010560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f355a9c4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6fa1010 con 0x55d0a700d290 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6f52e60 con 0x55d0a6fe0770 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6f50550 con 0x55d0a700d670 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a6fe0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a6fe0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37064/0 (socket says 172.21.15.31:37064) 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 -- 172.21.15.31:0/445686900 learned_addr learned my addr 172.21.15.31:0/445686900 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555393640 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a7010020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3556395640 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a6fe0770 0x55d0a7010560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 -- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 msgr2=0x55d0a7010020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a7010020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 -- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a6fe0770 msgr2=0x55d0a7010560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a6fe0770 0x55d0a7010560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 -- 172.21.15.31:0/445686900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a6f43b00 con 0x55d0a700d290 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f3555b94640 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a6fe0230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3548001e80 tx=0x7f3548011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77faeed716c2d276 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f35337fe640 1 -- 172.21.15.31:0/445686900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35480035f0 con 0x55d0a700d290 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f35337fe640 1 -- 172.21.15.31:0/445686900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548003750 con 0x55d0a700d290 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.560+0000 7f35337fe640 1 -- 172.21.15.31:0/445686900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548003ae0 con 0x55d0a700d290 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 -- 172.21.15.31:0/445686900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 msgr2=0x55d0a6fe0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a6fe0230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3548001e80 tx=0x7f3548011000 comp rx=0 tx=0).stop 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 -- 172.21.15.31:0/445686900 shutdown_connections 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a6fe0770 0x55d0a7010560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a7010020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 --2- 172.21.15.31:0/445686900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a6fe0230 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 -- 172.21.15.31:0/445686900 >> 172.21.15.31:0/445686900 conn(0x55d0a6fe24a0 msgr2=0x55d0a702b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 -- 172.21.15.31:0/445686900 shutdown_connections 2024-01-19T04:56:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.561+0000 7f355a9c4740 1 -- 172.21.15.31:0/445686900 wait complete. 2024-01-19T04:56:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.562+0000 7f355a9c4740 1 -- start start 2024-01-19T04:56:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.562+0000 7f355a9c4740 1 Processor -- start 2024-01-19T04:56:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d290 0x55d0a70111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d670 0x55d0a703aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a700f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f355a9c4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6f4be40 con 0x55d0a7008c70 2024-01-19T04:56:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6f05250 con 0x55d0a700d670 2024-01-19T04:56:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6f04ef0 con 0x55d0a700d290 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f355a9c4740 10 client.0 _ll_register_callbacks cb 0x55d0a6f09720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a700f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a700f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37072/0 (socket says 172.21.15.31:37072) 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 -- 172.21.15.31:0/871426510 learned_addr learned my addr 172.21.15.31:0/871426510 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3555393640 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d670 0x55d0a703aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3555b94640 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d290 0x55d0a70111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 -- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d290 msgr2=0x55d0a70111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d290 0x55d0a70111c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 -- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d670 msgr2=0x55d0a703aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d670 0x55d0a703aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 -- 172.21.15.31:0/871426510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354800f040 con 0x55d0a7008c70 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3556395640 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a700f460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f354c007de0 tx=0x7f354c012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a6a36727cba7e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.563+0000 7f3530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3530ff9640 1 -- 172.21.15.31:0/871426510 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c009df0 con 0x55d0a7008c70 2024-01-19T04:56:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3530ff9640 1 -- 172.21.15.31:0/871426510 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f354c018470 con 0x55d0a7008c70 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0a6f44cd0 con 0x55d0a7008c70 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3530ff9640 1 -- 172.21.15.31:0/871426510 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f354c018630 con 0x55d0a7008c70 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 10 client.5406 fetch_fsmap learned FSMap version 14 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55d0a6f4a4c0 con 0x55d0a7008c70 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3530ff9640 1 -- 172.21.15.31:0/871426510 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0a6f44cd0 con 0x55d0a7008c70 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3530ff9640 1 -- 172.21.15.31:0/871426510 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f354c014040 con 0x55d0a7008c70 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 10 client.5406 fetch_fsmap finished waiting for FSMap version 14 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 10 client.5406 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 20 client.5406 populate_metadata read hostname 'smithi031' 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a703b000 0x55d0a7011830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 4 client.5406 mds_command: new command op to 5259 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 --> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0a6f449f0 con 0x55d0a703b000 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3555b94640 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a703b000 0x55d0a7011830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3530ff9640 10 client.5406 ms_handle_connect on v2:172.21.15.31:6834/968278604 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.564+0000 7f3555b94640 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a703b000 0x55d0a7011830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4517641cf12bb3b server_cookie=53307d005bfcbf93 in_seq=0 out_seq=0 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f3530ff9640 1 -- 172.21.15.31:0/871426510 <== mds.0 v2:172.21.15.31:6834/968278604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0a6f449f0 con 0x55d0a703b000 2024-01-19T04:56:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f3530ff9640 10 client.5406 handle_command_reply: tid=0 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 client.5406 shutdown 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 msgr2=0x55d0a700f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a700f460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f354c007de0 tx=0x7f354c012610 comp rx=0 tx=0).stop 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 shutdown_connections 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a703b000 0x55d0a7011830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a700f460 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d670 0x55d0a703aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 --2- 172.21.15.31:0/871426510 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d290 0x55d0a70111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f3530ff9640 0 client.5406 ms_handle_reset on v2:172.21.15.31:6834/968278604 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 >> 172.21.15.31:0/871426510 conn(0x55d0a6fe0770 msgr2=0x55d0a702edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 shutdown_connections 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.565+0000 7f355a9c4740 1 -- 172.21.15.31:0/871426510 wait complete. 2024-01-19T04:56:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.566+0000 7f355a9c4740 20 client.5406 trim_cache size 0 max 16384 2024-01-19T04:56:30.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.581+0000 7f355a9c4740 1 Processor -- start 2024-01-19T04:56:30.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.581+0000 7f355a9c4740 1 -- start start 2024-01-19T04:56:30.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a7008c70 0x55d0a6fe7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a7007bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a704d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f355a9c4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6eca300 con 0x55d0a700d290 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6ed3390 con 0x55d0a7008c70 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a7044f60 con 0x55d0a700d670 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f3555393640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a7007bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f3555393640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a7007bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37074/0 (socket says 172.21.15.31:37074) 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f3555393640 1 -- 172.21.15.31:0/2843722358 learned_addr learned my addr 172.21.15.31:0/2843722358 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f3555b94640 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a7008c70 0x55d0a6fe7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f3555393640 1 -- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 msgr2=0x55d0a704d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.582+0000 7f3556395640 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a704d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3555393640 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a704d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3555393640 1 -- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a7008c70 msgr2=0x55d0a6fe7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3555393640 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a7008c70 0x55d0a6fe7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3555393640 1 -- 172.21.15.31:0/2843722358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a6ee7150 con 0x55d0a700d290 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3556395640 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a704d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3555b94640 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a7008c70 0x55d0a6fe7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3555393640 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a7007bd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f35400774a0 tx=0x7f35400ac040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e73b0ef210187b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3532ffd640 1 -- 172.21.15.31:0/2843722358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354007f0f0 con 0x55d0a700d290 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3532ffd640 1 -- 172.21.15.31:0/2843722358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35400832a0 con 0x55d0a700d290 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f3532ffd640 1 -- 172.21.15.31:0/2843722358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540081070 con 0x55d0a700d290 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 -- 172.21.15.31:0/2843722358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 msgr2=0x55d0a7007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a7007bd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f35400774a0 tx=0x7f35400ac040 comp rx=0 tx=0).stop 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 -- 172.21.15.31:0/2843722358 shutdown_connections 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a704d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a700d290 0x55d0a7007bd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 --2- 172.21.15.31:0/2843722358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a7008c70 0x55d0a6fe7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 -- 172.21.15.31:0/2843722358 >> 172.21.15.31:0/2843722358 conn(0x55d0a6fe0770 msgr2=0x55d0a6fdb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 -- 172.21.15.31:0/2843722358 shutdown_connections 2024-01-19T04:56:30.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.583+0000 7f355a9c4740 1 -- 172.21.15.31:0/2843722358 wait complete. 2024-01-19T04:56:30.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.584+0000 7f355a9c4740 1 -- start start 2024-01-19T04:56:30.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.584+0000 7f355a9c4740 1 Processor -- start 2024-01-19T04:56:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a7011bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d290 0x55d0a7011020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f355a9c4740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a6ffbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f355a9c4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6fe44a0 con 0x55d0a7008c70 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a6fe73c0 con 0x55d0a700d290 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f355a9c4740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a70115f0 con 0x55d0a700d670 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f355a9c4740 10 client.0 _ll_register_callbacks cb 0x7f354c009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a7011bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a7011bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37088/0 (socket says 172.21.15.31:37088) 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 -- 172.21.15.31:0/559574916 learned_addr learned my addr 172.21.15.31:0/559574916 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555393640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d290 0x55d0a7011020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3556395640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a6ffbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 -- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 msgr2=0x55d0a6ffbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a6ffbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 -- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d290 msgr2=0x55d0a7011020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d290 0x55d0a7011020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 -- 172.21.15.31:0/559574916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35500cc9b0 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3556395640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a6ffbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3555b94640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a7011bc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3548001570 tx=0x7f3548000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b7c8ed27297c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3530ff9640 1 -- 172.21.15.31:0/559574916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548028040 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.585+0000 7f3530ff9640 1 -- 172.21.15.31:0/559574916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f354802d5b0 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3548003750 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f3530ff9640 1 -- 172.21.15.31:0/559574916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f354800e600 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 10 client.5412 fetch_fsmap learned FSMap version 14 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55d0a6f0c940 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f3530ff9640 1 -- 172.21.15.31:0/559574916 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548003750 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f3530ff9640 1 -- 172.21.15.31:0/559574916 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3548034060 con 0x55d0a7008c70 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 10 client.5412 fetch_fsmap finished waiting for FSMap version 14 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 10 client.5412 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-19T04:56:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 20 client.5412 populate_metadata read hostname 'smithi031' 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a7094cc0 0x55d0a70950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 4 client.5412 mds_command: new command op to 5259 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 --> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d0a6c4b6c0 con 0x55d0a7094cc0 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f3555393640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a7094cc0 0x55d0a70950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f3530ff9640 10 client.5412 ms_handle_connect on v2:172.21.15.31:6834/968278604 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.586+0000 7f3555393640 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a7094cc0 0x55d0a70950a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1243f0540d1d23 server_cookie=23bce2887c85fcea in_seq=0 out_seq=0 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3530ff9640 1 -- 172.21.15.31:0/559574916 <== mds.0 v2:172.21.15.31:6834/968278604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d0a6c4b6c0 con 0x55d0a7094cc0 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3530ff9640 10 client.5412 handle_command_reply: tid=0 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 client.5412 shutdown 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 msgr2=0x55d0a7011bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a7011bc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3548001570 tx=0x7f3548000bc0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 shutdown_connections 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:6834/968278604,v1:172.21.15.31:6835/968278604] conn(0x55d0a7094cc0 0x55d0a70950a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55d0a700d670 0x55d0a6ffbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0a700d290 0x55d0a7011020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3530ff9640 0 client.5412 ms_handle_reset on v2:172.21.15.31:6834/968278604 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 --2- 172.21.15.31:0/559574916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a7008c70 0x55d0a7011bc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 >> 172.21.15.31:0/559574916 conn(0x55d0a6fe0770 msgr2=0x55d0a6fe3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 shutdown_connections 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 1 -- 172.21.15.31:0/559574916 wait complete. 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f355a9c4740 20 client.5412 trim_cache size 0 max 16384 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "16b8b5c3-25df-408f-9c77-cd32e346926d", 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5259, 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-01-19T04:56:30.593 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-01-19T04:56:30.594 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-01-19T04:56:30.594 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 4.6568399899999999, 2024-01-19T04:56:30.594 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 14, 2024-01-19T04:56:30.594 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 34, 2024-01-19T04:56:30.594 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 34, 2024-01-19T04:56:30.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.59251326 2024-01-19T04:56:30.594 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f353406af00 msgr2=0x7f353406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f353406af00 0x7f353406d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3548001660 tx=0x7f3548009040 comp rx=0 tx=0).stop 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d1fe0 msgr2=0x7f35500c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d1fe0 0x7f35500c9d20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3540000c00 tx=0x7f3540004730 comp rx=0 tx=0).stop 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 shutdown_connections 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f353406af00 0x7f353406d3c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f355015d700 0x7f3550150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35500d1fe0 0x7f35500c9d20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 --2- 172.21.15.31:0/3110041977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35500d11b0 0x7f35500c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.587+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 >> 172.21.15.31:0/3110041977 conn(0x7f35500d32f0 msgr2=0x7f355015b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.588+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 shutdown_connections 2024-01-19T04:56:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.588+0000 7f3557df8640 1 -- 172.21.15.31:0/3110041977 wait complete. 2024-01-19T04:56:30.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:56:30.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.698+0000 7f268bd5a640 1 Processor -- start 2024-01-19T04:56:30.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.698+0000 7f268bd5a640 1 -- start start 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268bd5a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f26840b1cc0 0x7f26840b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268bd5a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f268406e9d0 0x7f268406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268bd5a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26840b2660 con 0x7f2684144070 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268bd5a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2684144450 con 0x7f268406e9d0 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268bd5a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26840ced30 con 0x7f26840b1cc0 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268a2f7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f268406e9d0 0x7f268406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268a2f7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f268406e9d0 0x7f268406ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37094/0 (socket says 172.21.15.31:37094) 2024-01-19T04:56:30.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268a2f7640 1 -- 172.21.15.31:0/2555239746 learned_addr learned my addr 172.21.15.31:0/2555239746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f26892f5640 1 --2- 172.21.15.31:0/2555239746 >> v2:172.21.15.137:3300/0 conn(0x7f26840b1cc0 0x7f26840b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268a2f7640 1 -- 172.21.15.31:0/2555239746 >> v2:172.21.15.137:3300/0 conn(0x7f26840b1cc0 msgr2=0x7f26840b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268a2f7640 1 --2- 172.21.15.31:0/2555239746 >> v2:172.21.15.137:3300/0 conn(0x7f26840b1cc0 0x7f26840b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268a2f7640 1 -- 172.21.15.31:0/2555239746 >> v1:172.21.15.137:6789/0 conn(0x7f2684144070 legacy=0x7f26841445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:30.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.699+0000 7f268a2f7640 1 -- 172.21.15.31:0/2555239746 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840cf010 con 0x7f268406e9d0 2024-01-19T04:56:30.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.700+0000 7f268a2f7640 1 --2- 172.21.15.31:0/2555239746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f268406e9d0 0x7f268406ee40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2684072bd0 tx=0x7f267c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c160a757db957ca3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.700+0000 7f2688af4640 1 -- 172.21.15.31:0/2555239746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c02ca80 con 0x7f268406e9d0 2024-01-19T04:56:30.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.700+0000 7f2688af4640 1 -- 172.21.15.31:0/2555239746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c02cbe0 con 0x7f268406e9d0 2024-01-19T04:56:30.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.700+0000 7f2688af4640 1 -- 172.21.15.31:0/2555239746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0343f0 con 0x7f268406e9d0 2024-01-19T04:56:30.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 -- 172.21.15.31:0/2555239746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f268406e9d0 msgr2=0x7f268406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2555239746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f268406e9d0 0x7f268406ee40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2684072bd0 tx=0x7f267c02bee0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 -- 172.21.15.31:0/2555239746 shutdown_connections 2024-01-19T04:56:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2555239746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f268406e9d0 0x7f268406ee40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2555239746 >> v2:172.21.15.137:3300/0 conn(0x7f26840b1cc0 0x7f26840b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 -- 172.21.15.31:0/2555239746 >> 172.21.15.31:0/2555239746 conn(0x7f268406d750 msgr2=0x7f268406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 -- 172.21.15.31:0/2555239746 shutdown_connections 2024-01-19T04:56:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.701+0000 7f268bd5a640 1 -- 172.21.15.31:0/2555239746 wait complete. 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.702+0000 7f268bd5a640 1 Processor -- start 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.702+0000 7f268bd5a640 1 -- start start 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268bd5a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f268406e9d0 0x7f2684149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268bd5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26840b1cc0 0x7f2684147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268bd5a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2684144070 0x7f268415def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268bd5a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26841498d0 con 0x7f2684144070 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268bd5a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26841482f0 con 0x7f26840b1cc0 2024-01-19T04:56:30.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268bd5a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26841485d0 con 0x7f268406e9d0 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2684144070 0x7f268415def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2684144070 0x7f268415def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37104/0 (socket says 172.21.15.31:37104) 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 -- 172.21.15.31:0/2636691236 learned_addr learned my addr 172.21.15.31:0/2636691236 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f26892f5640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26840b1cc0 0x7f2684147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f2689af6640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f268406e9d0 0x7f2684149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 -- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f268406e9d0 msgr2=0x7f2684149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f268406e9d0 0x7f2684149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 -- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26840b1cc0 msgr2=0x7f2684147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26840b1cc0 0x7f2684147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 -- 172.21.15.31:0/2636691236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c009b30 con 0x7f2684144070 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.703+0000 7f268a2f7640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2684144070 0x7f268415def0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f267c036040 tx=0x7f267c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.704+0000 7f267a7fc640 1 -- 172.21.15.31:0/2636691236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c009510 con 0x7f2684144070 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.704+0000 7f267a7fc640 1 -- 172.21.15.31:0/2636691236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c03e450 con 0x7f2684144070 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.704+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26841488b0 con 0x7f2684144070 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.704+0000 7f267a7fc640 1 -- 172.21.15.31:0/2636691236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c03e7e0 con 0x7f2684144070 2024-01-19T04:56:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.704+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2684148e70 con 0x7f2684144070 2024-01-19T04:56:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.705+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268406edb0 con 0x7f2684144070 2024-01-19T04:56:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.705+0000 7f267a7fc640 1 -- 172.21.15.31:0/2636691236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f267c02cd50 con 0x7f2684144070 2024-01-19T04:56:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.705+0000 7f267a7fc640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f266806aed0 0x7f266806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.706+0000 7f267a7fc640 1 -- 172.21.15.31:0/2636691236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f267c0ac160 con 0x7f2684144070 2024-01-19T04:56:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.706+0000 7f2689af6640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f266806aed0 0x7f266806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.706+0000 7f2689af6640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f266806aed0 0x7f266806d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2680004ea0 tx=0x7f2680009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:30.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.710+0000 7f267a7fc640 1 -- 172.21.15.31:0/2636691236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f267c076fa0 con 0x7f2684144070 2024-01-19T04:56:30.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.925+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f26840696e0 con 0x7f2684144070 2024-01-19T04:56:30.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.926+0000 7f267a7fc640 1 -- 172.21.15.31:0/2636691236 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f267c07a5c0 con 0x7f2684144070 2024-01-19T04:56:30.925 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f266806aed0 msgr2=0x7f266806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f266806aed0 0x7f266806d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2680004ea0 tx=0x7f2680009040 comp rx=0 tx=0).stop 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2684144070 msgr2=0x7f268415def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2684144070 0x7f268415def0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f267c036040 tx=0x7f267c004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 shutdown_connections 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f266806aed0 0x7f266806d390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2684144070 0x7f268415def0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26840b1cc0 0x7f2684147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 --2- 172.21.15.31:0/2636691236 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f268406e9d0 0x7f2684149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 >> 172.21.15.31:0/2636691236 conn(0x7f268406d750 msgr2=0x7f268406ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 shutdown_connections 2024-01-19T04:56:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:30.928+0000 7f268bd5a640 1 -- 172.21.15.31:0/2636691236 wait complete. 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:56:30.938 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:30.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:30.939 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:56:30.953 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:56:30.953 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:56:30.953 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42702sec preferred_lft 42702sec 2024-01-19T04:56:30.954 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c57:fcff:feba:867a/64 scope link 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::98a9:56ff:fef7:e409/64 scope link 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:30.955 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:56:30.956 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:30.956 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::90b1:87ff:fefd:d7a/64 scope link 2024-01-19T04:56:30.956 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:30.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:30.956 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:56:31.019 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:56:31.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.021 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:56:31.077 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:56:31.077 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:56:31.077 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:56:31.077 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:56:31.077 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:56:31.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.078 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:56:31.078 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:56:31.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:56:31.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:56:31.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:56:31.078 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:56:31.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.172 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:56:31.186 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-19T04:56:31.186 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:56:31.186 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:31.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-01-19T04:56:31.254 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-19T04:56:31.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:56:31.255 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:31.308 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:31.309 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:56:31.309 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:31.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:56:31.431 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:56:31.487 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:56:31.487 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:56:31.487 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:56:31.487 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:56:31.487 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:56:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:31.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.490 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:56:31.541 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:56:31.542 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-19T04:56:31.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:56:31.611 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:56:31.666 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:56:31.667 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:56:31.668 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:31.669 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:31.669 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:31.669 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:31.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.669 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:56:31.670 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-01-19T04:56:31.670+0000 7f2fb7141480 -1 init, newargv = 0x5614bfc51830 newargc=15 2024-01-19T04:56:31.670 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[50945]: starting ceph client 2024-01-19T04:56:31.674 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[50945]: starting fuse 2024-01-19T04:56:31.684 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:56:31.684 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:56:31.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:56:31.685 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-01-19T04:56:31.686 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:56:31.804 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.50945.asok 2024-01-19T04:56:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:56:31.814 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50945.asok 2024-01-19T04:56:31.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50945.asok status 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "50945", 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5424, 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 5424 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3800397387 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3800397387 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.5424 192.168.0.1:0/3800397387", 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/3800397387", 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 14, 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 34, 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:56:31.953 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:56:31.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.959 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:56:31.959 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:31.974 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:56:31.974 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:31.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:31.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:32.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-01-19T04:56:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.188+0000 7f009d2c2640 1 Processor -- start 2024-01-19T04:56:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.188+0000 7f009d2c2640 1 -- start start 2024-01-19T04:56:32.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f009d2c2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 0x7f00980cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f009d2c2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f009806c940 0x7f00980cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f009d2c2640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009806bb40 con 0x7f009806c560 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f009d2c2640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00980cfeb0 con 0x7f009806c940 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f009d2c2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00980d00a0 con 0x7f009809a160 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f0096ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f009806c560 0x7f0098099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58920/0 (socket says 172.21.15.31:58920) 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f0096ffd640 1 -- 172.21.15.31:0/1185402869 learned_addr learned my addr 172.21.15.31:0/1185402869 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569364767 0 0) 0x7f009806bb40 con 0x7f009806c560 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00977fe640 1 --2- 172.21.15.31:0/1185402869 >> v2:172.21.15.137:3300/0 conn(0x7f009806c940 0x7f00980cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0084003520 con 0x7f009806c560 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f008ffff640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 0x7f00980cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3187242375 0 0) 0x7f0084003520 con 0x7f009806c560 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 msgr2=0x7f00980cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 0x7f00980cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 >> v2:172.21.15.137:3300/0 conn(0x7f009806c940 msgr2=0x7f00980cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 --2- 172.21.15.31:0/1185402869 >> v2:172.21.15.137:3300/0 conn(0x7f009806c940 0x7f00980cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00980d0380 con 0x7f009806c560 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.189+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f0088001670 con 0x7f009806c560 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 >> v1:172.21.15.31:6789/0 conn(0x7f009806c560 legacy=0x7f0098099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00967fc640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0084003970 0x7f0084003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00967fc640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00840047c0 0x7f0084004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00967fc640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 0x7f0084005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00980d00a0 con 0x7f00840047c0 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008400b980 con 0x7f0084003970 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008400baf0 con 0x7f009809a160 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f008ffff640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00840047c0 0x7f0084004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f0096ffd640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0084003970 0x7f0084003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f00977fe640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 0x7f0084005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f008ffff640 1 -- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 msgr2=0x7f0084005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f008ffff640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f009809a160 0x7f0084005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f008ffff640 1 -- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0084003970 msgr2=0x7f0084003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f008ffff640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0084003970 0x7f0084003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.190+0000 7f008ffff640 1 -- 172.21.15.31:0/1185402869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084003520 con 0x7f00840047c0 2024-01-19T04:56:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.191+0000 7f008ffff640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00840047c0 0x7f0084004c40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f008000ba30 tx=0x7f00840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd9ef419c71c494 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.191+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080032640 con 0x7f00840047c0 2024-01-19T04:56:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.191+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00800327a0 con 0x7f00840047c0 2024-01-19T04:56:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.191+0000 7f00967fc640 1 -- 172.21.15.31:0/1185402869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080032b30 con 0x7f00840047c0 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.191+0000 7f009d2c2640 1 -- 172.21.15.31:0/1185402869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00840047c0 msgr2=0x7f0084004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.191+0000 7f009d2c2640 1 --2- 172.21.15.31:0/1185402869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00840047c0 0x7f0084004c40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f008000ba30 tx=0x7f00840057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.191+0000 7f00977fe640 1 -- 172.21.15.31:0/1185402869 reap_dead start 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.192+0000 7f009d2c2640 1 -- 172.21.15.31:0/1185402869 shutdown_connections 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.192+0000 7f009d2c2640 1 -- 172.21.15.31:0/1185402869 >> 172.21.15.31:0/1185402869 conn(0x7f00980b2bb0 msgr2=0x7f00980b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.192+0000 7f009d2c2640 1 -- 172.21.15.31:0/1185402869 shutdown_connections 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.192+0000 7f009d2c2640 1 -- 172.21.15.31:0/1185402869 wait complete. 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.193+0000 7f009d2c2640 1 Processor -- start 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.193+0000 7f009d2c2640 1 -- start start 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.193+0000 7f009d2c2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00840047c0 0x7f00981e0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.193+0000 7f009d2c2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00980b6af0 0x7f00980b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.193+0000 7f009d2c2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00981e6c20 0x7f00981e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f009d2c2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981e75e0 con 0x7f00981e6c20 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f009d2c2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00980b4fa0 con 0x7f00840047c0 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f009d2c2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00980b4390 con 0x7f00980b6af0 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00981e6c20 0x7f00981e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00981e6c20 0x7f00981e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37130/0 (socket says 172.21.15.31:37130) 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 -- 172.21.15.31:0/489690345 learned_addr learned my addr 172.21.15.31:0/489690345 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f008ffff640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00980b6af0 0x7f00980b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f0096ffd640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00840047c0 0x7f00981e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 -- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00980b6af0 msgr2=0x7f00980b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00980b6af0 0x7f00980b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 -- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00840047c0 msgr2=0x7f00981e0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00840047c0 0x7f00981e0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 -- 172.21.15.31:0/489690345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00980b4670 con 0x7f00981e6c20 2024-01-19T04:56:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f00977fe640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00981e6c20 0x7f00981e70a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f009809ab80 tx=0x7f009000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f008f7fe640 1 -- 172.21.15.31:0/489690345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009000e040 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f008f7fe640 1 -- 172.21.15.31:0/489690345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009000f040 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00980b4950 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f008f7fe640 1 -- 172.21.15.31:0/489690345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090015670 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.194+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009809bb60 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.195+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00980696e0 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.196+0000 7f008f7fe640 1 -- 172.21.15.31:0/489690345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f00900157d0 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.196+0000 7f008f7fe640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f007806af00 0x7f007806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.196+0000 7f008f7fe640 1 -- 172.21.15.31:0/489690345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f009008a0b0 con 0x7f00981e6c20 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.196+0000 7f0096ffd640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f007806af00 0x7f007806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.197+0000 7f0096ffd640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f007806af00 0x7f007806d3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0088000fb0 tx=0x7f0088009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.200+0000 7f008f7fe640 1 -- 172.21.15.31:0/489690345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0090055010 con 0x7f00981e6c20 2024-01-19T04:56:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.377+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f00980b50e0 con 0x7f00981e6c20 2024-01-19T04:56:32.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.509+0000 7f008f7fe640 1 -- 172.21.15.31:0/489690345 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v35) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0090058630 con 0x7f00981e6c20 2024-01-19T04:56:32.509 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'data_new' created 2024-01-19T04:56:32.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f007806af00 msgr2=0x7f007806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f007806af00 0x7f007806d3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0088000fb0 tx=0x7f0088009040 comp rx=0 tx=0).stop 2024-01-19T04:56:32.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00981e6c20 msgr2=0x7f00981e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00981e6c20 0x7f00981e70a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f009809ab80 tx=0x7f009000f580 comp rx=0 tx=0).stop 2024-01-19T04:56:32.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 shutdown_connections 2024-01-19T04:56:32.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f007806af00 0x7f007806d3c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00981e6c20 0x7f00981e70a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00980b6af0 0x7f00980b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 --2- 172.21.15.31:0/489690345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00840047c0 0x7f00981e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 >> 172.21.15.31:0/489690345 conn(0x7f00980b2bb0 msgr2=0x7f00981e4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:32.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 shutdown_connections 2024-01-19T04:56:32.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.512+0000 7f009d2c2640 1 -- 172.21.15.31:0/489690345 wait complete. 2024-01-19T04:56:32.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-01-19T04:56:32.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.629+0000 7fddb5ae7640 1 Processor -- start 2024-01-19T04:56:32.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.629+0000 7fddb5ae7640 1 -- start start 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddb5ae7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddb00cf570 0x7fddb0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddb5ae7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fddb015d780 0x7fddb0161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddb5ae7640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb00cf9f0 con 0x7fddb015d3a0 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddb5ae7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb00cfcd0 con 0x7fddb00cf570 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddb5ae7640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb00cffb0 con 0x7fddb015d780 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaeffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddb00cf570 0x7fddb0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaf7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fddb015d3a0 0x7fddb006c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58934/0 (socket says 172.21.15.31:58934) 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaf7fe640 1 -- 172.21.15.31:0/1082578531 learned_addr learned my addr 172.21.15.31:0/1082578531 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaffff640 1 --2- 172.21.15.31:0/1082578531 >> v2:172.21.15.137:3300/0 conn(0x7fddb015d780 0x7fddb0161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaeffd640 1 -- 172.21.15.31:0/1082578531 >> v2:172.21.15.137:3300/0 conn(0x7fddb015d780 msgr2=0x7fddb0161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaeffd640 1 --2- 172.21.15.31:0/1082578531 >> v2:172.21.15.137:3300/0 conn(0x7fddb015d780 0x7fddb0161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaeffd640 1 -- 172.21.15.31:0/1082578531 >> v1:172.21.15.31:6789/0 conn(0x7fddb015d3a0 legacy=0x7fddb006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.630+0000 7fddaeffd640 1 -- 172.21.15.31:0/1082578531 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb00d0290 con 0x7fddb00cf570 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.631+0000 7fddaeffd640 1 --2- 172.21.15.31:0/1082578531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb0161150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fddb006d720 tx=0x7fdda402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2329da009808e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.631+0000 7fddae7fc640 1 -- 172.21.15.31:0/1082578531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4031830 con 0x7fddb00cf570 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.631+0000 7fddae7fc640 1 -- 172.21.15.31:0/1082578531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda4031990 con 0x7fddb00cf570 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.631+0000 7fddae7fc640 1 -- 172.21.15.31:0/1082578531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4031d20 con 0x7fddb00cf570 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 -- 172.21.15.31:0/1082578531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 msgr2=0x7fddb0161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/1082578531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb0161150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fddb006d720 tx=0x7fdda402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 -- 172.21.15.31:0/1082578531 shutdown_connections 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/1082578531 >> v2:172.21.15.137:3300/0 conn(0x7fddb015d780 0x7fddb0161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/1082578531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb0161150 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 -- 172.21.15.31:0/1082578531 >> 172.21.15.31:0/1082578531 conn(0x7fddb00d1100 msgr2=0x7fddb00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 -- 172.21.15.31:0/1082578531 shutdown_connections 2024-01-19T04:56:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.632+0000 7fddb5ae7640 1 -- 172.21.15.31:0/1082578531 wait complete. 2024-01-19T04:56:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.633+0000 7fddb5ae7640 1 Processor -- start 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.633+0000 7fddb5ae7640 1 -- start start 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddb5ae7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb00c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddb5ae7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb015d3a0 0x7fddb0153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddb5ae7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddb015d780 0x7fddb00ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddb5ae7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb01569c0 con 0x7fddb00cf570 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddb5ae7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb0156ca0 con 0x7fddb015d3a0 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddb5ae7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb0156f80 con 0x7fddb015d780 2024-01-19T04:56:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb00c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb00c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37144/0 (socket says 172.21.15.31:37144) 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 -- 172.21.15.31:0/920075436 learned_addr learned my addr 172.21.15.31:0/920075436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaeffd640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb015d3a0 0x7fddb0153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaffff640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddb015d780 0x7fddb00ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 -- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddb015d780 msgr2=0x7fddb00ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddb015d780 0x7fddb00ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 -- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb015d3a0 msgr2=0x7fddb0153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb015d3a0 0x7fddb0153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 -- 172.21.15.31:0/920075436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda4009400 con 0x7fddb00cf570 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaeffd640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb015d3a0 0x7fddb0153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddaf7fe640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb00c75d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdda000c780 tx=0x7fdda000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fdd97fff640 1 -- 172.21.15.31:0/920075436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda000ce90 con 0x7fddb00cf570 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fdd97fff640 1 -- 172.21.15.31:0/920075436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda000a640 con 0x7fddb00cf570 2024-01-19T04:56:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb0157260 con 0x7fddb00cf570 2024-01-19T04:56:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.634+0000 7fdd97fff640 1 -- 172.21.15.31:0/920075436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda000aa10 con 0x7fddb00cf570 2024-01-19T04:56:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.635+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb0154500 con 0x7fddb00cf570 2024-01-19T04:56:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.635+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb0074900 con 0x7fddb00cf570 2024-01-19T04:56:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.636+0000 7fdd97fff640 1 -- 172.21.15.31:0/920075436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdda000ab70 con 0x7fddb00cf570 2024-01-19T04:56:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.636+0000 7fdd97fff640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd9c06aed0 0x7fdd9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.637+0000 7fdd97fff640 1 -- 172.21.15.31:0/920075436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6072+0+0 (secure 0 0 0) 0x7fdda0089010 con 0x7fddb00cf570 2024-01-19T04:56:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.637+0000 7fddaeffd640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd9c06aed0 0x7fdd9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.637+0000 7fddaeffd640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd9c06aed0 0x7fdd9c06d390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdda4036000 tx=0x7fdda403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.641+0000 7fdd97fff640 1 -- 172.21.15.31:0/920075436 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdda0057800 con 0x7fddb00cf570 2024-01-19T04:56:32.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:32.865+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7fddb0067030 con 0x7fddb00cf570 2024-01-19T04:56:33.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.511+0000 7fdd97fff640 1 -- 172.21.15.31:0/920075436 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 6 to fsmap v15) v1 ==== 131+0+0 (secure 0 0 0) 0x7fdda0001840 con 0x7fddb00cf570 2024-01-19T04:56:33.511 INFO:teuthology.orchestra.run.smithi031.stderr:added data pool 6 to fsmap 2024-01-19T04:56:33.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd9c06aed0 msgr2=0x7fdd9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd9c06aed0 0x7fdd9c06d390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdda4036000 tx=0x7fdda403c040 comp rx=0 tx=0).stop 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 msgr2=0x7fddb00c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb00c75d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdda000c780 tx=0x7fdda000cc40 comp rx=0 tx=0).stop 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 shutdown_connections 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd9c06aed0 0x7fdd9c06d390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fddb015d780 0x7fddb00ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb015d3a0 0x7fddb0153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 --2- 172.21.15.31:0/920075436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb00cf570 0x7fddb00c75d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 >> 172.21.15.31:0/920075436 conn(0x7fddb00d1100 msgr2=0x7fddb0155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.515+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 shutdown_connections 2024-01-19T04:56:33.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.516+0000 7fddb5ae7640 1 -- 172.21.15.31:0/920075436 wait complete. 2024-01-19T04:56:33.537 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:33.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.644+0000 7f9a0b382640 1 Processor -- start 2024-01-19T04:56:33.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.644+0000 7f9a0b382640 1 -- start start 2024-01-19T04:56:33.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0b382640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0b382640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a040b4d00 0x7f9a040b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0b382640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a04160df0 con 0x7f9a040d2e50 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0b382640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040b56b0 con 0x7f9a040b4d00 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0b382640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040b7300 con 0x7f9a04160a10 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0991f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a040b4d00 0x7f9a040b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0991f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a040b4d00 0x7f9a040b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37156/0 (socket says 172.21.15.31:37156) 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.645+0000 7f9a0991f640 1 -- 172.21.15.31:0/1282621126 learned_addr learned my addr 172.21.15.31:0/1282621126 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.646+0000 7f9a0911e640 1 --2- 172.21.15.31:0/1282621126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.646+0000 7f9a0991f640 1 -- 172.21.15.31:0/1282621126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 msgr2=0x7f9a040b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.646+0000 7f9a0991f640 1 --2- 172.21.15.31:0/1282621126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.646+0000 7f9a0991f640 1 -- 172.21.15.31:0/1282621126 >> v1:172.21.15.137:6789/0 conn(0x7f9a040d2e50 legacy=0x7f9a040d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.646+0000 7f9a0991f640 1 -- 172.21.15.31:0/1282621126 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040d1db0 con 0x7f9a040b4d00 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.646+0000 7f9a0991f640 1 --2- 172.21.15.31:0/1282621126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a040b5170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009870 tx=0x7f9a0002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3a5dea70efdd92 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f99fbfff640 1 -- 172.21.15.31:0/1282621126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0002f840 con 0x7f9a040b4d00 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f99fbfff640 1 -- 172.21.15.31:0/1282621126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0002f9a0 con 0x7f9a040b4d00 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f99fbfff640 1 -- 172.21.15.31:0/1282621126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0002fd30 con 0x7f9a040b4d00 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 -- 172.21.15.31:0/1282621126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 msgr2=0x7f9a040b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 --2- 172.21.15.31:0/1282621126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a040b5170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009870 tx=0x7f9a0002ba20 comp rx=0 tx=0).stop 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 -- 172.21.15.31:0/1282621126 shutdown_connections 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 --2- 172.21.15.31:0/1282621126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a040b5170 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 --2- 172.21.15.31:0/1282621126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 -- 172.21.15.31:0/1282621126 >> 172.21.15.31:0/1282621126 conn(0x7f9a04072310 msgr2=0x7f9a040cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 -- 172.21.15.31:0/1282621126 shutdown_connections 2024-01-19T04:56:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.647+0000 7f9a0b382640 1 -- 172.21.15.31:0/1282621126 wait complete. 2024-01-19T04:56:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.648+0000 7f9a0b382640 1 Processor -- start 2024-01-19T04:56:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.648+0000 7f9a0b382640 1 -- start start 2024-01-19T04:56:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0b382640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a04157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0b382640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a040d2e50 0x7f9a041567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0b382640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0b382640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040cd130 con 0x7f9a040b4d00 2024-01-19T04:56:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0b382640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041572e0 con 0x7f9a040d2e50 2024-01-19T04:56:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0b382640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041575c0 con 0x7f9a04160a10 2024-01-19T04:56:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0911e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a04157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0911e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a04157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37172/0 (socket says 172.21.15.31:37172) 2024-01-19T04:56:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0911e640 1 -- 172.21.15.31:0/3552315192 learned_addr learned my addr 172.21.15.31:0/3552315192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0991f640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.649+0000 7f9a0891d640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a040d2e50 0x7f9a041567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0911e640 1 -- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 msgr2=0x7f9a040cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0911e640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0911e640 1 -- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a040d2e50 msgr2=0x7f9a041567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0911e640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a040d2e50 0x7f9a041567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0911e640 1 -- 172.21.15.31:0/3552315192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000094d0 con 0x7f9a040b4d00 2024-01-19T04:56:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0911e640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a04157920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99fc007ec0 tx=0x7f99fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f99f9ffb640 1 -- 172.21.15.31:0/3552315192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc014620 con 0x7f9a040b4d00 2024-01-19T04:56:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f99f9ffb640 1 -- 172.21.15.31:0/3552315192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc0147a0 con 0x7f9a040b4d00 2024-01-19T04:56:33.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a040ccae0 con 0x7f9a040b4d00 2024-01-19T04:56:33.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f99f9ffb640 1 -- 172.21.15.31:0/3552315192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc014b50 con 0x7f9a040b4d00 2024-01-19T04:56:33.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.650+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a04157e60 con 0x7f9a040b4d00 2024-01-19T04:56:33.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.651+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0415f9b0 con 0x7f9a040b4d00 2024-01-19T04:56:33.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.651+0000 7f99f9ffb640 1 -- 172.21.15.31:0/3552315192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f99fc014cb0 con 0x7f9a040b4d00 2024-01-19T04:56:33.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.652+0000 7f99f9ffb640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.652+0000 7f99f9ffb640 1 -- 172.21.15.31:0/3552315192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f99fc08a150 con 0x7f9a040b4d00 2024-01-19T04:56:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.652+0000 7f9a0891d640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.653+0000 7f9a0891d640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9a040c98a0 tx=0x7f99f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.656+0000 7f99f9ffb640 1 -- 172.21.15.31:0/3552315192 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f99fc054f20 con 0x7f9a040b4d00 2024-01-19T04:56:33.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.871+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a040696e0 con 0x7f9a040b4d00 2024-01-19T04:56:33.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.871+0000 7f99f9ffb640 1 -- 172.21.15.31:0/3552315192 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f99fc058540 con 0x7f9a040b4d00 2024-01-19T04:56:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:33.871 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:56:25.278275+0000","modified":"2024-01-19T04:56:32.866830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5259":{"gid":5259,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/968278604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":968278604},{"type":"v1","addr":"172.21.15.31:6835","nonce":968278604}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-19T04:56:33.871 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 msgr2=0x7f99e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9a040c98a0 tx=0x7f99f4009040 comp rx=0 tx=0).stop 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 msgr2=0x7f9a04157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a04157920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99fc007ec0 tx=0x7f99fc00e4b0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 shutdown_connections 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a04160a10 0x7f9a040cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a040d2e50 0x7f9a041567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 --2- 172.21.15.31:0/3552315192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a040b4d00 0x7f9a04157920 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 >> 172.21.15.31:0/3552315192 conn(0x7f9a04072310 msgr2=0x7f9a040d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 shutdown_connections 2024-01-19T04:56:33.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.873+0000 7f9a0b382640 1 -- 172.21.15.31:0/3552315192 wait complete. 2024-01-19T04:56:33.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.985+0000 7f07d9105640 1 Processor -- start 2024-01-19T04:56:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d9105640 1 -- start start 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d9105640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07d4160bb0 0x7f07d40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d9105640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f07d40cfd10 0x7f07d40d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d9105640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d40d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d9105640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d40d06b0 con 0x7f07d4160bb0 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d9105640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d40b66b0 con 0x7f07d40cfd10 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d9105640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40d18c0 con 0x7f07d40d0ef0 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d2d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07d4160bb0 0x7f07d40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d2d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07d4160bb0 0x7f07d40b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37186/0 (socket says 172.21.15.31:37186) 2024-01-19T04:56:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.986+0000 7f07d2d9d640 1 -- 172.21.15.31:0/1921659443 learned_addr learned my addr 172.21.15.31:0/1921659443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d259c640 1 --2- 172.21.15.31:0/1921659443 >> v2:172.21.15.137:3300/0 conn(0x7f07d40cfd10 0x7f07d40d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d359e640 1 --2- 172.21.15.31:0/1921659443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d40d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d2d9d640 1 -- 172.21.15.31:0/1921659443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 msgr2=0x7f07d40d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d2d9d640 1 --2- 172.21.15.31:0/1921659443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d40d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d2d9d640 1 -- 172.21.15.31:0/1921659443 >> v2:172.21.15.137:3300/0 conn(0x7f07d40cfd10 msgr2=0x7f07d40d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d2d9d640 1 --2- 172.21.15.31:0/1921659443 >> v2:172.21.15.137:3300/0 conn(0x7f07d40cfd10 0x7f07d40d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d2d9d640 1 -- 172.21.15.31:0/1921659443 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40d1b90 con 0x7f07d4160bb0 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.987+0000 7f07d2d9d640 1 --2- 172.21.15.31:0/1921659443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d40b6170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f07d408d990 tx=0x7f07cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de1794b04411fe server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d1d9b640 1 -- 172.21.15.31:0/1921659443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc031870 con 0x7f07d4160bb0 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d1d9b640 1 -- 172.21.15.31:0/1921659443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc0319d0 con 0x7f07d4160bb0 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d1d9b640 1 -- 172.21.15.31:0/1921659443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc031d60 con 0x7f07d4160bb0 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 -- 172.21.15.31:0/1921659443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 msgr2=0x7f07d40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 --2- 172.21.15.31:0/1921659443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d40b6170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f07d408d990 tx=0x7f07cc02b910 comp rx=0 tx=0).stop 2024-01-19T04:56:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 -- 172.21.15.31:0/1921659443 shutdown_connections 2024-01-19T04:56:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 --2- 172.21.15.31:0/1921659443 >> v2:172.21.15.137:3300/0 conn(0x7f07d40cfd10 0x7f07d40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 --2- 172.21.15.31:0/1921659443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d40d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 --2- 172.21.15.31:0/1921659443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d40b6170 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 -- 172.21.15.31:0/1921659443 >> 172.21.15.31:0/1921659443 conn(0x7f07d409c600 msgr2=0x7f07d40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 -- 172.21.15.31:0/1921659443 shutdown_connections 2024-01-19T04:56:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.988+0000 7f07d9105640 1 -- 172.21.15.31:0/1921659443 wait complete. 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 Processor -- start 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 -- start start 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d40cfd10 0x7f07d41f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d4159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d4151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40cb330 con 0x7f07d4160bb0 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40ca7b0 con 0x7f07d40cfd10 2024-01-19T04:56:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d9105640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40caa90 con 0x7f07d40d0ef0 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d4151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d4151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37202/0 (socket says 172.21.15.31:37202) 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.990+0000 7f07d359e640 1 -- 172.21.15.31:0/2813906082 learned_addr learned my addr 172.21.15.31:0/2813906082 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d359e640 1 -- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 msgr2=0x7f07d4159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d259c640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d4159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d2d9d640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d40cfd10 0x7f07d41f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d359e640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d4159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d359e640 1 -- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d40cfd10 msgr2=0x7f07d41f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d359e640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d40cfd10 0x7f07d41f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d359e640 1 -- 172.21.15.31:0/2813906082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0094d0 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d2d9d640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d40cfd10 0x7f07d41f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d359e640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d4151390 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007e60 tx=0x7f07c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07bb7fe640 1 -- 172.21.15.31:0/2813906082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4014420 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07bb7fe640 1 -- 172.21.15.31:0/2813906082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c40145a0 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d40cad70 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07bb7fe640 1 -- 172.21.15.31:0/2813906082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4014950 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.991+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d40cbe80 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.992+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d40696e0 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.993+0000 7f07bb7fe640 1 -- 172.21.15.31:0/2813906082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f07c4014ab0 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.993+0000 7f07bb7fe640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07c006aed0 0x7f07c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.993+0000 7f07bb7fe640 1 -- 172.21.15.31:0/2813906082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f07c408a100 con 0x7f07d4160bb0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.993+0000 7f07d2d9d640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07c006aed0 0x7f07c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.994+0000 7f07d2d9d640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07c006aed0 0x7f07c006d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07cc036000 tx=0x7f07cc03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:33.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:33.997+0000 7f07bb7fe640 1 -- 172.21.15.31:0/2813906082 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f07c4054e50 con 0x7f07d4160bb0 2024-01-19T04:56:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.176+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07d40c6ee0 con 0x7f07d4160bb0 2024-01-19T04:56:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.177+0000 7f07bb7fe640 1 -- 172.21.15.31:0/2813906082 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18326 (secure 0 0 0) 0x7f07c4058470 con 0x7f07d4160bb0 2024-01-19T04:56:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:34.177 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:32.866625+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07c006aed0 msgr2=0x7f07c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07c006aed0 0x7f07c006d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07cc036000 tx=0x7f07cc03c040 comp rx=0 tx=0).stop 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 msgr2=0x7f07d4151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d4151390 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007e60 tx=0x7f07c400e4b0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 shutdown_connections 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f07c006aed0 0x7f07c006d390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d4160bb0 0x7f07d4151390 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f07d40d0ef0 0x7f07d4159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 --2- 172.21.15.31:0/2813906082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d40cfd10 0x7f07d41f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 >> 172.21.15.31:0/2813906082 conn(0x7f07d409c600 msgr2=0x7f07d415b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 shutdown_connections 2024-01-19T04:56:34.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.179+0000 7f07d9105640 1 -- 172.21.15.31:0/2813906082 wait complete. 2024-01-19T04:56:34.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:34.190 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2024-01-19T04:56:34.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:34.209 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2024-01-19T04:56:34.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-01-19T04:56:34.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.417+0000 7fef7ec1e640 1 Processor -- start 2024-01-19T04:56:34.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.417+0000 7fef7ec1e640 1 -- start start 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7ec1e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fef7815b6b0 0x7fef780d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7ec1e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef780cd2e0 0x7fef780cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7ec1e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780cce50 con 0x7fef780ceee0 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7ec1e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780d03b0 con 0x7fef7815b6b0 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7ec1e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780cc240 con 0x7fef780cd2e0 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7c9ba640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fef7815b6b0 0x7fef780d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef77fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef780cd2e0 0x7fef780cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7d1bb640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fef780ceee0 0x7fef780b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:45900/0 (socket says 172.21.15.31:45900) 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7c9ba640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fef7815b6b0 0x7fef780d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:48178/0 (socket says 172.21.15.31:48178) 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7d1bb640 1 -- 172.21.15.31:0/3795738789 learned_addr learned my addr 172.21.15.31:0/3795738789 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7c9ba640 1 -- 172.21.15.31:0/3795738789 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef780cd2e0 msgr2=0x7fef780cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7c9ba640 1 --2- 172.21.15.31:0/3795738789 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef780cd2e0 0x7fef780cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7c9ba640 1 -- 172.21.15.31:0/3795738789 >> v1:172.21.15.137:6789/0 conn(0x7fef780ceee0 legacy=0x7fef780b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.418+0000 7fef7c9ba640 1 -- 172.21.15.31:0/3795738789 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780cc520 con 0x7fef7815b6b0 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.419+0000 7fef7c9ba640 1 --2- 172.21.15.31:0/3795738789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef7815b6b0 0x7fef780d1c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009790 tx=0x7fef6c02b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e033d80ed68f97cd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.419+0000 7fef777fe640 1 -- 172.21.15.31:0/3795738789 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c031890 con 0x7fef7815b6b0 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef777fe640 1 -- 172.21.15.31:0/3795738789 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c0319f0 con 0x7fef7815b6b0 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef777fe640 1 -- 172.21.15.31:0/3795738789 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c031d80 con 0x7fef7815b6b0 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 -- 172.21.15.31:0/3795738789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef7815b6b0 msgr2=0x7fef780d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/3795738789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef7815b6b0 0x7fef780d1c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009790 tx=0x7fef6c02b930 comp rx=0 tx=0).stop 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 -- 172.21.15.31:0/3795738789 shutdown_connections 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/3795738789 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef780cd2e0 0x7fef780cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/3795738789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef7815b6b0 0x7fef780d1c20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 -- 172.21.15.31:0/3795738789 >> 172.21.15.31:0/3795738789 conn(0x7fef780d1100 msgr2=0x7fef780d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 -- 172.21.15.31:0/3795738789 shutdown_connections 2024-01-19T04:56:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.420+0000 7fef7ec1e640 1 -- 172.21.15.31:0/3795738789 wait complete. 2024-01-19T04:56:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.421+0000 7fef7ec1e640 1 Processor -- start 2024-01-19T04:56:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.421+0000 7fef7ec1e640 1 -- start start 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7ec1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef780cd2e0 0x7fef78157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7ec1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780ceee0 0x7fef780c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7ec1e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef7815b6b0 0x7fef780b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7ec1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781579a0 con 0x7fef780ceee0 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7ec1e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780c7150 con 0x7fef780cd2e0 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7ec1e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780ca030 con 0x7fef7815b6b0 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780ceee0 0x7fef780c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780ceee0 0x7fef780c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37208/0 (socket says 172.21.15.31:37208) 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef77fff640 1 -- 172.21.15.31:0/684771774 learned_addr learned my addr 172.21.15.31:0/684771774 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7c9ba640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef780cd2e0 0x7fef78157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef7d1bb640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef7815b6b0 0x7fef780b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef77fff640 1 -- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef7815b6b0 msgr2=0x7fef780b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.422+0000 7fef77fff640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef7815b6b0 0x7fef780b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef77fff640 1 -- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef780cd2e0 msgr2=0x7fef78157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef77fff640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef780cd2e0 0x7fef78157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef77fff640 1 -- 172.21.15.31:0/684771774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0093a0 con 0x7fef780ceee0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef77fff640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780ceee0 0x7fef780c6c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e9b0 tx=0x7fef6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef757fa640 1 -- 172.21.15.31:0/684771774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64009670 con 0x7fef780ceee0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef757fa640 1 -- 172.21.15.31:0/684771774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6400c9a0 con 0x7fef780ceee0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef757fa640 1 -- 172.21.15.31:0/684771774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6400cd50 con 0x7fef780ceee0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef780c5e50 con 0x7fef780ceee0 2024-01-19T04:56:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.423+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef780c6410 con 0x7fef780ceee0 2024-01-19T04:56:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.424+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78074900 con 0x7fef780ceee0 2024-01-19T04:56:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.424+0000 7fef757fa640 1 -- 172.21.15.31:0/684771774 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fef64013470 con 0x7fef780ceee0 2024-01-19T04:56:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.425+0000 7fef757fa640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fef5c06af00 0x7fef5c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.425+0000 7fef757fa640 1 -- 172.21.15.31:0/684771774 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fef640899a0 con 0x7fef780ceee0 2024-01-19T04:56:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.425+0000 7fef7c9ba640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fef5c06af00 0x7fef5c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.426+0000 7fef7c9ba640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fef5c06af00 0x7fef5c06d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef6c036000 tx=0x7fef6c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.429+0000 7fef757fa640 1 -- 172.21.15.31:0/684771774 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fef640546f0 con 0x7fef780ceee0 2024-01-19T04:56:34.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.678+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fef780b2780 con 0x7fef780ceee0 2024-01-19T04:56:34.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.681+0000 7fef757fa640 1 -- 172.21.15.31:0/684771774 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v10) v1 ==== 210+0+167 (secure 0 0 0) 0x7fef640210c0 con 0x7fef780ceee0 2024-01-19T04:56:34.680 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:56:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:56:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow rw" 2024-01-19T04:56:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:56:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-01-19T04:56:34.681 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:56:34.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fef5c06af00 msgr2=0x7fef5c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fef5c06af00 0x7fef5c06d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef6c036000 tx=0x7fef6c03c040 comp rx=0 tx=0).stop 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780ceee0 msgr2=0x7fef780c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780ceee0 0x7fef780c6c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e9b0 tx=0x7fef6400ee70 comp rx=0 tx=0).stop 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 shutdown_connections 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fef5c06af00 0x7fef5c06d3c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fef7815b6b0 0x7fef780b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780ceee0 0x7fef780c6c10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 --2- 172.21.15.31:0/684771774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef780cd2e0 0x7fef78157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 >> 172.21.15.31:0/684771774 conn(0x7fef780d1100 msgr2=0x7fef7815c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:34.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 shutdown_connections 2024-01-19T04:56:34.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.683+0000 7fef7ec1e640 1 -- 172.21.15.31:0/684771774 wait complete. 2024-01-19T04:56:34.693 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:34.693 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:34.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.851+0000 7fd2f1808640 1 Processor -- start 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.851+0000 7fd2f1808640 1 -- start start 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2f1808640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd2ec0cfd10 0x7fd2ec0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2f1808640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2ec0d0ef0 0x7fd2ec0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2f1808640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec0d06b0 con 0x7fd2ec160bb0 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2f1808640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ec0d18c0 con 0x7fd2ec0d0ef0 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2f1808640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ec0d1b90 con 0x7fd2ec0cfd10 2024-01-19T04:56:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2eb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2ec0d0ef0 0x7fd2ec0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2eaffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd2ec160bb0 0x7fd2ec0b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58942/0 (socket says 172.21.15.31:58942) 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2eaffd640 1 -- 172.21.15.31:0/320805410 learned_addr learned my addr 172.21.15.31:0/320805410 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2ea7fc640 1 --2- 172.21.15.31:0/320805410 >> v2:172.21.15.137:3300/0 conn(0x7fd2ec0cfd10 0x7fd2ec0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2eb7fe640 1 -- 172.21.15.31:0/320805410 >> v2:172.21.15.137:3300/0 conn(0x7fd2ec0cfd10 msgr2=0x7fd2ec0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2eb7fe640 1 --2- 172.21.15.31:0/320805410 >> v2:172.21.15.137:3300/0 conn(0x7fd2ec0cfd10 0x7fd2ec0d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2eb7fe640 1 -- 172.21.15.31:0/320805410 >> v1:172.21.15.31:6789/0 conn(0x7fd2ec160bb0 legacy=0x7fd2ec0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2eb7fe640 1 -- 172.21.15.31:0/320805410 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec0d1e70 con 0x7fd2ec0d0ef0 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2e9ffb640 1 -- 172.21.15.31:0/320805410 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277977501 0 0) 0x7fd2ec0d06b0 con 0x7fd2ec160bb0 2024-01-19T04:56:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.852+0000 7fd2ea7fc640 1 --2- 172.21.15.31:0/320805410 >> v2:172.21.15.137:3300/0 conn(0x7fd2ec0cfd10 0x7fd2ec0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:34.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.853+0000 7fd2eb7fe640 1 --2- 172.21.15.31:0/320805410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec0d0ef0 0x7fd2ec0d1380 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec08d990 tx=0x7fd2dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57299ca17d599b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.853+0000 7fd2e9ffb640 1 -- 172.21.15.31:0/320805410 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc02cc80 con 0x7fd2ec0d0ef0 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.853+0000 7fd2e9ffb640 1 -- 172.21.15.31:0/320805410 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc02cde0 con 0x7fd2ec0d0ef0 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.853+0000 7fd2e9ffb640 1 -- 172.21.15.31:0/320805410 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc034660 con 0x7fd2ec0d0ef0 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 -- 172.21.15.31:0/320805410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec0d0ef0 msgr2=0x7fd2ec0d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 --2- 172.21.15.31:0/320805410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec0d0ef0 0x7fd2ec0d1380 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec08d990 tx=0x7fd2dc02bed0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 -- 172.21.15.31:0/320805410 shutdown_connections 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 --2- 172.21.15.31:0/320805410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec0d0ef0 0x7fd2ec0d1380 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 --2- 172.21.15.31:0/320805410 >> v2:172.21.15.137:3300/0 conn(0x7fd2ec0cfd10 0x7fd2ec0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 -- 172.21.15.31:0/320805410 >> 172.21.15.31:0/320805410 conn(0x7fd2ec09c600 msgr2=0x7fd2ec0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 -- 172.21.15.31:0/320805410 shutdown_connections 2024-01-19T04:56:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.854+0000 7fd2f1808640 1 -- 172.21.15.31:0/320805410 wait complete. 2024-01-19T04:56:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.855+0000 7fd2f1808640 1 Processor -- start 2024-01-19T04:56:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.855+0000 7fd2f1808640 1 -- start start 2024-01-19T04:56:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2f1808640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd2ec0cfd10 0x7fd2ec1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2f1808640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2ec0d0ef0 0x7fd2ec06b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2f1808640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec160bb0 0x7fd2ec06b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2f1808640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1f6a70 con 0x7fd2ec160bb0 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2f1808640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1f39a0 con 0x7fd2ec0d0ef0 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2f1808640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1f3c80 con 0x7fd2ec0cfd10 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec160bb0 0x7fd2ec06b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec160bb0 0x7fd2ec06b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37232/0 (socket says 172.21.15.31:37232) 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 -- 172.21.15.31:0/3080085502 learned_addr learned my addr 172.21.15.31:0/3080085502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2ea7fc640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2ec0d0ef0 0x7fd2ec06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eaffd640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd2ec0cfd10 0x7fd2ec1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 -- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd2ec0cfd10 msgr2=0x7fd2ec1f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd2ec0cfd10 0x7fd2ec1f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 -- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2ec0d0ef0 msgr2=0x7fd2ec06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2ec0d0ef0 0x7fd2ec06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 -- 172.21.15.31:0/3080085502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc009a70 con 0x7fd2ec160bb0 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eaffd640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd2ec0cfd10 0x7fd2ec1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.856+0000 7fd2eb7fe640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec160bb0 0x7fd2ec06b840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009e80 tx=0x7fd2dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.857+0000 7fd2e3fff640 1 -- 172.21.15.31:0/3080085502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc03e610 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.857+0000 7fd2e3fff640 1 -- 172.21.15.31:0/3080085502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc03e770 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.857+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec1f3f60 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.857+0000 7fd2e3fff640 1 -- 172.21.15.31:0/3080085502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc03eb60 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.857+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1f73d0 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.858+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec1f6bb0 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.858+0000 7fd2e3fff640 1 -- 172.21.15.31:0/3080085502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd2dc03b050 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.858+0000 7fd2e3fff640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd2cc06aed0 0x7fd2cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.859+0000 7fd2e3fff640 1 -- 172.21.15.31:0/3080085502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd2dc0ac1f0 con 0x7fd2ec160bb0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.859+0000 7fd2eaffd640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd2cc06aed0 0x7fd2cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.859+0000 7fd2eaffd640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd2cc06aed0 0x7fd2cc06d390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4000f60 tx=0x7fd2e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:34.863+0000 7fd2e3fff640 1 -- 172.21.15.31:0/3080085502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd2ec1f6bb0 con 0x7fd2ec160bb0 2024-01-19T04:56:35.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.045+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2ec1f6bb0 con 0x7fd2ec160bb0 2024-01-19T04:56:35.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.046+0000 7fd2e3fff640 1 -- 172.21.15.31:0/3080085502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7fd2ec1f6bb0 con 0x7fd2ec160bb0 2024-01-19T04:56:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:35.046 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd2cc06aed0 msgr2=0x7fd2cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd2cc06aed0 0x7fd2cc06d390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4000f60 tx=0x7fd2e4009040 comp rx=0 tx=0).stop 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec160bb0 msgr2=0x7fd2ec06b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec160bb0 0x7fd2ec06b840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009e80 tx=0x7fd2dc004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 shutdown_connections 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd2cc06aed0 0x7fd2cc06d390 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ec160bb0 0x7fd2ec06b840 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2ec0d0ef0 0x7fd2ec06b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 --2- 172.21.15.31:0/3080085502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd2ec0cfd10 0x7fd2ec1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 >> 172.21.15.31:0/3080085502 conn(0x7fd2ec09c600 msgr2=0x7fd2ec06c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 shutdown_connections 2024-01-19T04:56:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.048+0000 7fd2f1808640 1 -- 172.21.15.31:0/3080085502 wait complete. 2024-01-19T04:56:35.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:56:23.785927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:56:24.240600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-01-19T04:56:32.396495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:35.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:35.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.162+0000 7fe45ef3a640 1 Processor -- start 2024-01-19T04:56:35.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.162+0000 7fe45ef3a640 1 -- start start 2024-01-19T04:56:35.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe45ef3a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe45815b6b0 0x7fe4580d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe45ef3a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe4580ce090 0x7fe4580b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe45ef3a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580cce50 con 0x7fe4580cd2e0 2024-01-19T04:56:35.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe45ef3a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580d03b0 con 0x7fe45815b6b0 2024-01-19T04:56:35.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe45ef3a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580cc240 con 0x7fe4580ce090 2024-01-19T04:56:35.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe45ccd6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe45815b6b0 0x7fe4580d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe457fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe4580cd2e0 0x7fe4580cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58954/0 (socket says 172.21.15.31:58954) 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.163+0000 7fe457fff640 1 -- 172.21.15.31:0/3047280880 learned_addr learned my addr 172.21.15.31:0/3047280880 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.164+0000 7fe45ccd6640 1 -- 172.21.15.31:0/3047280880 >> v2:172.21.15.137:3300/0 conn(0x7fe4580ce090 msgr2=0x7fe4580b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.164+0000 7fe45d4d7640 1 --2- 172.21.15.31:0/3047280880 >> v2:172.21.15.137:3300/0 conn(0x7fe4580ce090 0x7fe4580b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.164+0000 7fe4577fe640 1 -- 172.21.15.31:0/3047280880 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144424261 0 0) 0x7fe4580cce50 con 0x7fe4580cd2e0 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.164+0000 7fe45ccd6640 1 --2- 172.21.15.31:0/3047280880 >> v2:172.21.15.137:3300/0 conn(0x7fe4580ce090 0x7fe4580b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.164+0000 7fe45ccd6640 1 -- 172.21.15.31:0/3047280880 >> v1:172.21.15.31:6789/0 conn(0x7fe4580cd2e0 legacy=0x7fe4580cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.164+0000 7fe45ccd6640 1 -- 172.21.15.31:0/3047280880 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580cc520 con 0x7fe45815b6b0 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.164+0000 7fe45ccd6640 1 --2- 172.21.15.31:0/3047280880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45815b6b0 0x7fe4580d1c20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe4580d1090 tx=0x7fe45002bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf629ddcc94c8a63 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe4577fe640 1 -- 172.21.15.31:0/3047280880 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45002cd40 con 0x7fe45815b6b0 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe4577fe640 1 -- 172.21.15.31:0/3047280880 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450034440 con 0x7fe45815b6b0 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe4577fe640 1 -- 172.21.15.31:0/3047280880 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45002cea0 con 0x7fe45815b6b0 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3047280880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45815b6b0 msgr2=0x7fe4580d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3047280880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45815b6b0 0x7fe4580d1c20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe4580d1090 tx=0x7fe45002bf90 comp rx=0 tx=0).stop 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3047280880 shutdown_connections 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3047280880 >> v2:172.21.15.137:3300/0 conn(0x7fe4580ce090 0x7fe4580b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3047280880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45815b6b0 0x7fe4580d1c20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3047280880 >> 172.21.15.31:0/3047280880 conn(0x7fe4580d1100 msgr2=0x7fe4580d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3047280880 shutdown_connections 2024-01-19T04:56:35.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.165+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3047280880 wait complete. 2024-01-19T04:56:35.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.166+0000 7fe45ef3a640 1 Processor -- start 2024-01-19T04:56:35.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.166+0000 7fe45ef3a640 1 -- start start 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45ef3a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe4580cd2e0 0x7fe458157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45ef3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4580ce090 0x7fe4580c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45ef3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe45815b6b0 0x7fe458154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45ef3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580b53b0 con 0x7fe4580ce090 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45ef3a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580b50b0 con 0x7fe45815b6b0 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45ef3a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe458157b70 con 0x7fe4580cd2e0 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe457fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4580ce090 0x7fe4580c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe457fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4580ce090 0x7fe4580c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37246/0 (socket says 172.21.15.31:37246) 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe457fff640 1 -- 172.21.15.31:0/3746559448 learned_addr learned my addr 172.21.15.31:0/3746559448 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45d4d7640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe45815b6b0 0x7fe458154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.167+0000 7fe45ccd6640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe4580cd2e0 0x7fe458157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe457fff640 1 -- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe4580cd2e0 msgr2=0x7fe458157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe457fff640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe4580cd2e0 0x7fe458157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe457fff640 1 -- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe45815b6b0 msgr2=0x7fe458154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe457fff640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe45815b6b0 0x7fe458154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe457fff640 1 -- 172.21.15.31:0/3746559448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450009a70 con 0x7fe4580ce090 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe457fff640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4580ce090 0x7fe4580c7a30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00c4d0 tx=0x7fe44c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe4557fa640 1 -- 172.21.15.31:0/3746559448 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0074e0 con 0x7fe4580ce090 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe4557fa640 1 -- 172.21.15.31:0/3746559448 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c007660 con 0x7fe4580ce090 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe4557fa640 1 -- 172.21.15.31:0/3746559448 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c004a20 con 0x7fe4580ce090 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe458157e50 con 0x7fe4580ce090 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.168+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe458158130 con 0x7fe4580ce090 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.169+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4580b5560 con 0x7fe4580ce090 2024-01-19T04:56:35.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.170+0000 7fe4557fa640 1 -- 172.21.15.31:0/3746559448 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe44c004b80 con 0x7fe4580ce090 2024-01-19T04:56:35.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.170+0000 7fe4557fa640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe44406aed0 0x7fe44406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.170+0000 7fe4557fa640 1 -- 172.21.15.31:0/3746559448 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe44c0888b0 con 0x7fe4580ce090 2024-01-19T04:56:35.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.170+0000 7fe45ccd6640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe44406aed0 0x7fe44406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.171+0000 7fe45ccd6640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe44406aed0 0x7fe44406d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe450036000 tx=0x7fe450030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.174+0000 7fe4557fa640 1 -- 172.21.15.31:0/3746559448 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe4580b5560 con 0x7fe4580ce090 2024-01-19T04:56:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.355+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe458074900 con 0x7fe4580ce090 2024-01-19T04:56:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.355+0000 7fe4557fa640 1 -- 172.21.15.31:0/3746559448 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe44c018020 con 0x7fe4580ce090 2024-01-19T04:56:35.355 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe44406aed0 msgr2=0x7fe44406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe44406aed0 0x7fe44406d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe450036000 tx=0x7fe450030000 comp rx=0 tx=0).stop 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4580ce090 msgr2=0x7fe4580c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4580ce090 0x7fe4580c7a30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00c4d0 tx=0x7fe44c00c990 comp rx=0 tx=0).stop 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 shutdown_connections 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe44406aed0 0x7fe44406d390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe45815b6b0 0x7fe458154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4580ce090 0x7fe4580c7a30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 --2- 172.21.15.31:0/3746559448 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe4580cd2e0 0x7fe458157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 >> 172.21.15.31:0/3746559448 conn(0x7fe4580d1100 msgr2=0x7fe45815dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 shutdown_connections 2024-01-19T04:56:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.357+0000 7fe45ef3a640 1 -- 172.21.15.31:0/3746559448 wait complete. 2024-01-19T04:56:35.367 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:35.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.472+0000 7ff7d67de640 1 Processor -- start 2024-01-19T04:56:35.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.472+0000 7ff7d67de640 1 -- start start 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7d67de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 0x7ff7d0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7d67de640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d00cf9f0 con 0x7ff7d015d3a0 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7d67de640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d00cfcd0 con 0x7ff7d015d780 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7d67de640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d00cffb0 con 0x7ff7d00cf570 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7cffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff7d015d3a0 0x7ff7d006c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58964/0 (socket says 172.21.15.31:58964) 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7cffff640 1 -- 172.21.15.31:0/2784075183 learned_addr learned my addr 172.21.15.31:0/2784075183 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7cf7fe640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 0x7ff7d0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994692999 0 0) 0x7ff7d00cf9f0 con 0x7ff7d015d3a0 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b8003520 con 0x7ff7d015d3a0 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 61779981 0 0) 0x7ff7b8003520 con 0x7ff7d015d3a0 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 msgr2=0x7ff7d0161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 0x7ff7d0161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 >> v1:172.21.15.137:6789/0 conn(0x7ff7d015d780 legacy=0x7ff7d0167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d00d0290 con 0x7ff7d015d3a0 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7ff7c00017f0 con 0x7ff7d015d3a0 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 >> v1:172.21.15.31:6789/0 conn(0x7ff7d015d3a0 legacy=0x7ff7d006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b80039a0 0x7ff7b8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b8004350 0x7ff7b8004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 0x7ff7b8005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d00cffb0 con 0x7ff7b80039a0 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b800b9b0 con 0x7ff7b8004350 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b800bb70 con 0x7ff7d00cf570 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.473+0000 7ff7cffff640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b80039a0 0x7ff7b8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7cf7fe640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b8004350 0x7ff7b8004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7d4d7b640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 0x7ff7b8005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7cffff640 1 -- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 msgr2=0x7ff7b8005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:35.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7cffff640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d00cf570 0x7ff7b8005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7cffff640 1 -- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b8004350 msgr2=0x7ff7b8004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7cffff640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b8004350 0x7ff7b8004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7cffff640 1 -- 172.21.15.31:0/2784075183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8003520 con 0x7ff7b80039a0 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.474+0000 7ff7cffff640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b80039a0 0x7ff7b8003d80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c240 tx=0x7ff7b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285fdde4ba42be43 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c002cad0 con 0x7ff7b80039a0 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c002cc30 con 0x7ff7b80039a0 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7ceffd640 1 -- 172.21.15.31:0/2784075183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0007070 con 0x7ff7b80039a0 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7d67de640 1 -- 172.21.15.31:0/2784075183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b80039a0 msgr2=0x7ff7b8003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7d67de640 1 --2- 172.21.15.31:0/2784075183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b80039a0 0x7ff7b8003d80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c240 tx=0x7ff7b80057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7d4d7b640 1 -- 172.21.15.31:0/2784075183 reap_dead start 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7d67de640 1 -- 172.21.15.31:0/2784075183 shutdown_connections 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7d67de640 1 -- 172.21.15.31:0/2784075183 >> 172.21.15.31:0/2784075183 conn(0x7ff7d00d1100 msgr2=0x7ff7d00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7d67de640 1 -- 172.21.15.31:0/2784075183 shutdown_connections 2024-01-19T04:56:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.475+0000 7ff7d67de640 1 -- 172.21.15.31:0/2784075183 wait complete. 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.476+0000 7ff7d67de640 1 Processor -- start 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d67de640 1 -- start start 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d67de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b80039a0 0x7ff7d00c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d67de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d015b4c0 0x7ff7d00c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d67de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7d00c50a0 0x7ff7d00c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d67de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d009ecb0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d67de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d0152a90 con 0x7ff7b80039a0 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d67de640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d0157d90 con 0x7ff7d015b4c0 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d4d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7d00c50a0 0x7ff7d00c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d4d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7d00c50a0 0x7ff7d00c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37264/0 (socket says 172.21.15.31:37264) 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.477+0000 7ff7d4d7b640 1 -- 172.21.15.31:0/1713876737 learned_addr learned my addr 172.21.15.31:0/1713876737 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7cffff640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b80039a0 0x7ff7d00c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7cf7fe640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d015b4c0 0x7ff7d00c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d4d7b640 1 -- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d015b4c0 msgr2=0x7ff7d00c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d4d7b640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d015b4c0 0x7ff7d00c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d4d7b640 1 -- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b80039a0 msgr2=0x7ff7d00c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d4d7b640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b80039a0 0x7ff7d00c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d4d7b640 1 -- 172.21.15.31:0/1713876737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d009e6f0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7cf7fe640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d015b4c0 0x7ff7d00c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d4d7b640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7d00c50a0 0x7ff7d00c5520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0071820 tx=0x7ff7c800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7ccff9640 1 -- 172.21.15.31:0/1713876737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8003b80 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7ccff9640 1 -- 172.21.15.31:0/1713876737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c8003d00 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7ccff9640 1 -- 172.21.15.31:0/1713876737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c800d9b0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d009e9d0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.478+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d0157a20 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.479+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d0157be0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.480+0000 7ff7ccff9640 1 -- 172.21.15.31:0/1713876737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff7c800db10 con 0x7ff7d00c50a0 2024-01-19T04:56:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.480+0000 7ff7ccff9640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff7bc06aed0 0x7ff7bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.480+0000 7ff7ccff9640 1 -- 172.21.15.31:0/1713876737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7ff7c80898e0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.480+0000 7ff7cffff640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff7bc06aed0 0x7ff7bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.481+0000 7ff7cffff640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff7bc06aed0 0x7ff7bc06d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0034000 tx=0x7ff7c0009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.484+0000 7ff7ccff9640 1 -- 172.21.15.31:0/1713876737 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff7c8054620 con 0x7ff7d00c50a0 2024-01-19T04:56:35.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.665+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7d01575a0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.665+0000 7ff7ccff9640 1 -- 172.21.15.31:0/1713876737 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7d01575a0 con 0x7ff7d00c50a0 2024-01-19T04:56:35.665 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff7bc06aed0 msgr2=0x7ff7bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff7bc06aed0 0x7ff7bc06d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0034000 tx=0x7ff7c0009030 comp rx=0 tx=0).stop 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7d00c50a0 msgr2=0x7ff7d00c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7d00c50a0 0x7ff7d00c5520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0071820 tx=0x7ff7c800ff70 comp rx=0 tx=0).stop 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 shutdown_connections 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff7bc06aed0 0x7ff7bc06d390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7d00c50a0 0x7ff7d00c5520 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7d015b4c0 0x7ff7d00c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 --2- 172.21.15.31:0/1713876737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7b80039a0 0x7ff7d00c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 >> 172.21.15.31:0/1713876737 conn(0x7ff7d00d1100 msgr2=0x7ff7d015f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:35.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 shutdown_connections 2024-01-19T04:56:35.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.667+0000 7ff7d67de640 1 -- 172.21.15.31:0/1713876737 wait complete. 2024-01-19T04:56:35.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.782+0000 7efe731b0640 1 Processor -- start 2024-01-19T04:56:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.782+0000 7efe731b0640 1 -- start start 2024-01-19T04:56:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe731b0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efe6c15d3a0 0x7efe6c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe731b0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c0cf9f0 con 0x7efe6c0cf570 2024-01-19T04:56:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe731b0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c0cfcd0 con 0x7efe6c161860 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe731b0640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6c0cffb0 con 0x7efe6c15d3a0 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efe6c0cf570 0x7efe6c161150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58968/0 (socket says 172.21.15.31:58968) 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6bfff640 1 -- 172.21.15.31:0/969263367 learned_addr learned my addr 172.21.15.31:0/969263367 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe70f4c640 1 --2- 172.21.15.31:0/969263367 >> v2:172.21.15.137:3300/0 conn(0x7efe6c15d3a0 0x7efe6c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293239102 0 0) 0x7efe6c0cf9f0 con 0x7efe6c0cf570 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe5c003520 con 0x7efe6c0cf570 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032070996 0 0) 0x7efe5c003520 con 0x7efe6c0cf570 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 >> v2:172.21.15.137:3300/0 conn(0x7efe6c15d3a0 msgr2=0x7efe6c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 --2- 172.21.15.31:0/969263367 >> v2:172.21.15.137:3300/0 conn(0x7efe6c15d3a0 0x7efe6c06c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 >> v1:172.21.15.137:6789/0 conn(0x7efe6c161860 legacy=0x7efe6c167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c0d0290 con 0x7efe6c0cf570 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7efe58001670 con 0x7efe6c0cf570 2024-01-19T04:56:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 >> v1:172.21.15.31:6789/0 conn(0x7efe6c0cf570 legacy=0x7efe6c161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c0047c0 0x7efe5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe5c0051d0 0x7efe5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0cffb0 con 0x7efe5c0039c0 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c00c120 con 0x7efe5c0047c0 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.783+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c00c2e0 con 0x7efe5c0051d0 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe70f4c640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe6bfff640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c0047c0 0x7efe5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe7174d640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe5c0051d0 0x7efe5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe70f4c640 1 -- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe5c0051d0 msgr2=0x7efe5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe70f4c640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe5c0051d0 0x7efe5c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe70f4c640 1 -- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c0047c0 msgr2=0x7efe5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe70f4c640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c0047c0 0x7efe5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe70f4c640 1 -- 172.21.15.31:0/969263367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c003520 con 0x7efe5c0039c0 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe7174d640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe5c0051d0 0x7efe5c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.784+0000 7efe7174d640 1 -- 172.21.15.31:0/969263367 reap_dead start 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe70f4c640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe5c003de0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efe60009a80 tx=0x7efe5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b824e3a21b4f778 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60005990 con 0x7efe5c0039c0 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60005af0 con 0x7efe5c0039c0 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe6b7fe640 1 -- 172.21.15.31:0/969263367 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60005ea0 con 0x7efe5c0039c0 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe731b0640 1 -- 172.21.15.31:0/969263367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 msgr2=0x7efe5c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe731b0640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe5c003de0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efe60009a80 tx=0x7efe5c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe731b0640 1 -- 172.21.15.31:0/969263367 shutdown_connections 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe731b0640 1 --2- 172.21.15.31:0/969263367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe5c003de0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe731b0640 1 -- 172.21.15.31:0/969263367 >> 172.21.15.31:0/969263367 conn(0x7efe6c0d1100 msgr2=0x7efe6c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe731b0640 1 -- 172.21.15.31:0/969263367 shutdown_connections 2024-01-19T04:56:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.785+0000 7efe731b0640 1 -- 172.21.15.31:0/969263367 wait complete. 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.786+0000 7efe731b0640 1 Processor -- start 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.786+0000 7efe731b0640 1 -- start start 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe731b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe6c14c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe731b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c0c78c0 0x7efe6c14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe731b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe6c09b070 0x7efe6c0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe731b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c14cb90 con 0x7efe5c0039c0 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe731b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c14d570 con 0x7efe6c0c78c0 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe731b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c09d0f0 con 0x7efe6c09b070 2024-01-19T04:56:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe70f4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe6c14c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe70f4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe6c14c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37290/0 (socket says 172.21.15.31:37290) 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.787+0000 7efe70f4c640 1 -- 172.21.15.31:0/2486409218 learned_addr learned my addr 172.21.15.31:0/2486409218 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe70f4c640 1 -- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe6c09b070 msgr2=0x7efe6c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe70f4c640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe6c09b070 0x7efe6c0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe6bfff640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c0c78c0 0x7efe6c14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe70f4c640 1 -- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c0c78c0 msgr2=0x7efe6c14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe70f4c640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c0c78c0 0x7efe6c14d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe70f4c640 1 -- 172.21.15.31:0/2486409218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c09d3d0 con 0x7efe5c0039c0 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe70f4c640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe6c14c650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efe60009ba0 tx=0x7efe60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe697fa640 1 -- 172.21.15.31:0/2486409218 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60005e70 con 0x7efe5c0039c0 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe697fa640 1 -- 172.21.15.31:0/2486409218 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60006030 con 0x7efe5c0039c0 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c09d6b0 con 0x7efe5c0039c0 2024-01-19T04:56:35.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe697fa640 1 -- 172.21.15.31:0/2486409218 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600063e0 con 0x7efe5c0039c0 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.788+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c09cd10 con 0x7efe5c0039c0 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.789+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c074900 con 0x7efe5c0039c0 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.789+0000 7efe697fa640 1 -- 172.21.15.31:0/2486409218 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efe60006540 con 0x7efe5c0039c0 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.790+0000 7efe697fa640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe5006af00 0x7efe5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.790+0000 7efe697fa640 1 -- 172.21.15.31:0/2486409218 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7efe600abc30 con 0x7efe5c0039c0 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.790+0000 7efe6bfff640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe5006af00 0x7efe5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.791+0000 7efe6bfff640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe5006af00 0x7efe5006d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe58001eb0 tx=0x7efe58000ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:35.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.794+0000 7efe697fa640 1 -- 172.21.15.31:0/2486409218 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efe60076a90 con 0x7efe5c0039c0 2024-01-19T04:56:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.976+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe6c067030 con 0x7efe5c0039c0 2024-01-19T04:56:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.976+0000 7efe697fa640 1 -- 172.21.15.31:0/2486409218 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe6004d020 con 0x7efe5c0039c0 2024-01-19T04:56:35.976 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.978+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe5006af00 msgr2=0x7efe5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.978+0000 7efe731b0640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe5006af00 0x7efe5006d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe58001eb0 tx=0x7efe58000ed0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.978+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 msgr2=0x7efe6c14c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.978+0000 7efe731b0640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe6c14c650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efe60009ba0 tx=0x7efe60004730 comp rx=0 tx=0).stop 2024-01-19T04:56:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 shutdown_connections 2024-01-19T04:56:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe5006af00 0x7efe5006d3c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe6c09b070 0x7efe6c0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c0c78c0 0x7efe6c14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 --2- 172.21.15.31:0/2486409218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe5c0039c0 0x7efe6c14c650 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 >> 172.21.15.31:0/2486409218 conn(0x7efe6c0d1100 msgr2=0x7efe6c161150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 shutdown_connections 2024-01-19T04:56:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:35.979+0000 7efe731b0640 1 -- 172.21.15.31:0/2486409218 wait complete. 2024-01-19T04:56:35.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-19T04:56:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.093+0000 7fe38b86b640 1 Processor -- start 2024-01-19T04:56:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.093+0000 7fe38b86b640 1 -- start start 2024-01-19T04:56:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe38b86b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d1c20 0x7fe3840d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe38b86b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe3840d3ea0 0x7fe3840b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe38b86b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3840cf160 con 0x7fe38415d720 2024-01-19T04:56:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe38b86b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3840ce8c0 con 0x7fe3840d3ea0 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe38b86b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840ceba0 con 0x7fe3840d1c20 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe389e08640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe3840d3ea0 0x7fe3840b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe389e08640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe3840d3ea0 0x7fe3840b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37300/0 (socket says 172.21.15.31:37300) 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.094+0000 7fe389e08640 1 -- 172.21.15.31:0/1505881165 learned_addr learned my addr 172.21.15.31:0/1505881165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.095+0000 7fe389e08640 1 -- 172.21.15.31:0/1505881165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d1c20 msgr2=0x7fe3840d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.095+0000 7fe388e06640 1 --2- 172.21.15.31:0/1505881165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d1c20 0x7fe3840d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.095+0000 7fe389e08640 1 --2- 172.21.15.31:0/1505881165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d1c20 0x7fe3840d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.095+0000 7fe389e08640 1 -- 172.21.15.31:0/1505881165 >> v1:172.21.15.137:6789/0 conn(0x7fe38415d720 legacy=0x7fe3840d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.095+0000 7fe389e08640 1 -- 172.21.15.31:0/1505881165 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840cee80 con 0x7fe3840d3ea0 2024-01-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.095+0000 7fe388e06640 1 --2- 172.21.15.31:0/1505881165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d1c20 0x7fe3840d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.095+0000 7fe389e08640 1 --2- 172.21.15.31:0/1505881165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3840d3ea0 0x7fe3840b3000 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe37402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fee7be0c4b12a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe383fff640 1 -- 172.21.15.31:0/1505881165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37402c9e0 con 0x7fe3840d3ea0 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe383fff640 1 -- 172.21.15.31:0/1505881165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37402cb40 con 0x7fe3840d3ea0 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe383fff640 1 -- 172.21.15.31:0/1505881165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740343f0 con 0x7fe3840d3ea0 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 -- 172.21.15.31:0/1505881165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3840d3ea0 msgr2=0x7fe3840b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 --2- 172.21.15.31:0/1505881165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3840d3ea0 0x7fe3840b3000 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe37402be40 comp rx=0 tx=0).stop 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 -- 172.21.15.31:0/1505881165 shutdown_connections 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 --2- 172.21.15.31:0/1505881165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3840d3ea0 0x7fe3840b3000 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 --2- 172.21.15.31:0/1505881165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d1c20 0x7fe3840d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 -- 172.21.15.31:0/1505881165 >> 172.21.15.31:0/1505881165 conn(0x7fe3840d3310 msgr2=0x7fe3840d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 -- 172.21.15.31:0/1505881165 shutdown_connections 2024-01-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.096+0000 7fe38b86b640 1 -- 172.21.15.31:0/1505881165 wait complete. 2024-01-19T04:56:36.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.097+0000 7fe38b86b640 1 Processor -- start 2024-01-19T04:56:36.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.097+0000 7fe38b86b640 1 -- start start 2024-01-19T04:56:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe38b86b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3840d1c20 0x7fe384156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe38b86b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d3ea0 0x7fe3840cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe38b86b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38415d720 0x7fe3841573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe38b86b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe384157250 con 0x7fe38415d720 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe38b86b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840c7e70 con 0x7fe3840d1c20 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe38b86b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840c8150 con 0x7fe3840d3ea0 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389e08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38415d720 0x7fe3841573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389e08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38415d720 0x7fe3841573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37308/0 (socket says 172.21.15.31:37308) 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389e08640 1 -- 172.21.15.31:0/4054197207 learned_addr learned my addr 172.21.15.31:0/4054197207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389607640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3840d1c20 0x7fe384156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe388e06640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d3ea0 0x7fe3840cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389e08640 1 -- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d3ea0 msgr2=0x7fe3840cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389e08640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d3ea0 0x7fe3840cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389e08640 1 -- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3840d1c20 msgr2=0x7fe384156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.098+0000 7fe389e08640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3840d1c20 0x7fe384156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe389e08640 1 -- 172.21.15.31:0/4054197207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe374009b30 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe389607640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3840d1c20 0x7fe384156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe388e06640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d3ea0 0x7fe3840cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe389e08640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38415d720 0x7fe3841573e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe374004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe381ffb640 1 -- 172.21.15.31:0/4054197207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37403e450 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe381ffb640 1 -- 172.21.15.31:0/4054197207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37403e5b0 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3840c8430 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe381ffb640 1 -- 172.21.15.31:0/4054197207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37403e960 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.099+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3840c89f0 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.100+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe384074900 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.101+0000 7fe381ffb640 1 -- 172.21.15.31:0/4054197207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe37403eac0 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.101+0000 7fe381ffb640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe37006af00 0x7fe37006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.101+0000 7fe381ffb640 1 -- 172.21.15.31:0/4054197207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe3740ac620 con 0x7fe38415d720 2024-01-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.101+0000 7fe389607640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe37006af00 0x7fe37006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.102+0000 7fe389607640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe37006af00 0x7fe37006d3c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe3840b5c10 tx=0x7fe37c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.105+0000 7fe381ffb640 1 -- 172.21.15.31:0/4054197207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe3740773d0 con 0x7fe38415d720 2024-01-19T04:56:36.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.284+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fe3840d1620 con 0x7fe38415d720 2024-01-19T04:56:36.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.285+0000 7fe381ffb640 1 -- 172.21.15.31:0/4054197207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7fe37407a9f0 con 0x7fe38415d720 2024-01-19T04:56:36.284 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:36.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe37006af00 msgr2=0x7fe37006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe37006af00 0x7fe37006d3c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe3840b5c10 tx=0x7fe37c009040 comp rx=0 tx=0).stop 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38415d720 msgr2=0x7fe3841573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38415d720 0x7fe3841573e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe374004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 shutdown_connections 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe37006af00 0x7fe37006d3c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38415d720 0x7fe3841573e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3840d3ea0 0x7fe3840cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 --2- 172.21.15.31:0/4054197207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3840d1c20 0x7fe384156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 >> 172.21.15.31:0/4054197207 conn(0x7fe3840d3310 msgr2=0x7fe3840b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 shutdown_connections 2024-01-19T04:56:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.287+0000 7fe38b86b640 1 -- 172.21.15.31:0/4054197207 wait complete. 2024-01-19T04:56:36.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:36.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.399+0000 7fdd65ec2640 1 Processor -- start 2024-01-19T04:56:36.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.400+0000 7fdd65ec2640 1 -- start start 2024-01-19T04:56:36.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.400+0000 7fdd65ec2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fdd6015d720 0x7fdd600d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.400+0000 7fdd65ec2640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600cf160 con 0x7fdd600d3ea0 2024-01-19T04:56:36.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.400+0000 7fdd65ec2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600ce8c0 con 0x7fdd600d1c20 2024-01-19T04:56:36.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.400+0000 7fdd65ec2640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd600ceba0 con 0x7fdd6015d720 2024-01-19T04:56:36.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5effd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd600d3ea0 0x7fdd600d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58984/0 (socket says 172.21.15.31:58984) 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5effd640 1 -- 172.21.15.31:0/371940372 learned_addr learned my addr 172.21.15.31:0/371940372 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5f7fe640 1 --2- 172.21.15.31:0/371940372 >> v2:172.21.15.137:3300/0 conn(0x7fdd6015d720 0x7fdd600d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5e7fc640 1 -- 172.21.15.31:0/371940372 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037262946 0 0) 0x7fdd600cf160 con 0x7fdd600d3ea0 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5e7fc640 1 -- 172.21.15.31:0/371940372 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd48003520 con 0x7fdd600d3ea0 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5f7fe640 1 -- 172.21.15.31:0/371940372 >> v1:172.21.15.137:6789/0 conn(0x7fdd600d1c20 legacy=0x7fdd600b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5f7fe640 1 -- 172.21.15.31:0/371940372 >> v1:172.21.15.31:6789/0 conn(0x7fdd600d3ea0 legacy=0x7fdd600d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.401+0000 7fdd5f7fe640 1 -- 172.21.15.31:0/371940372 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600cee80 con 0x7fdd6015d720 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd5f7fe640 1 --2- 172.21.15.31:0/371940372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd6015d720 0x7fdd600d0ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd600cf350 tx=0x7fdd5002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67474bde56703a5a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd5e7fc640 1 -- 172.21.15.31:0/371940372 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5002e070 con 0x7fdd6015d720 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd5e7fc640 1 -- 172.21.15.31:0/371940372 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd500315a0 con 0x7fdd6015d720 2024-01-19T04:56:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd5e7fc640 1 -- 172.21.15.31:0/371940372 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50031910 con 0x7fdd6015d720 2024-01-19T04:56:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd65ec2640 1 -- 172.21.15.31:0/371940372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd6015d720 msgr2=0x7fdd600d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/371940372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd6015d720 0x7fdd600d0ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd600cf350 tx=0x7fdd5002b840 comp rx=0 tx=0).stop 2024-01-19T04:56:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd65ec2640 1 -- 172.21.15.31:0/371940372 shutdown_connections 2024-01-19T04:56:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/371940372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd6015d720 0x7fdd600d0ee0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.402+0000 7fdd65ec2640 1 -- 172.21.15.31:0/371940372 >> 172.21.15.31:0/371940372 conn(0x7fdd600d3310 msgr2=0x7fdd600d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:36.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.403+0000 7fdd65ec2640 1 -- 172.21.15.31:0/371940372 shutdown_connections 2024-01-19T04:56:36.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.403+0000 7fdd65ec2640 1 -- 172.21.15.31:0/371940372 wait complete. 2024-01-19T04:56:36.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.404+0000 7fdd65ec2640 1 Processor -- start 2024-01-19T04:56:36.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.404+0000 7fdd65ec2640 1 -- start start 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd65ec2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd600d1c20 0x7fdd600c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd65ec2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd600d3ea0 0x7fdd601588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd65ec2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd6015d720 0x7fdd600b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd65ec2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60158df0 con 0x7fdd600d3ea0 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd65ec2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60159000 con 0x7fdd600d1c20 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd65ec2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601592e0 con 0x7fdd6015d720 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd600d3ea0 0x7fdd601588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd600d3ea0 0x7fdd601588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37314/0 (socket says 172.21.15.31:37314) 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 -- 172.21.15.31:0/619163955 learned_addr learned my addr 172.21.15.31:0/619163955 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5f7fe640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd600d1c20 0x7fdd600c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5ffff640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd6015d720 0x7fdd600b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 -- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd6015d720 msgr2=0x7fdd600b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd6015d720 0x7fdd600b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 -- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd600d1c20 msgr2=0x7fdd600c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd600d1c20 0x7fdd600c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 -- 172.21.15.31:0/619163955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50009400 con 0x7fdd600d3ea0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5f7fe640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd600d1c20 0x7fdd600c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd5effd640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd600d3ea0 0x7fdd601588b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c4d0 tx=0x7fdd5400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd47fff640 1 -- 172.21.15.31:0/619163955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54005ce0 con 0x7fdd600d3ea0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd47fff640 1 -- 172.21.15.31:0/619163955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd54005e40 con 0x7fdd600d3ea0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd47fff640 1 -- 172.21.15.31:0/619163955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54007270 con 0x7fdd600d3ea0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.405+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd601595c0 con 0x7fdd600d3ea0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.406+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd60159b80 con 0x7fdd600d3ea0 2024-01-19T04:56:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.406+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd600c7610 con 0x7fdd600d3ea0 2024-01-19T04:56:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.407+0000 7fdd47fff640 1 -- 172.21.15.31:0/619163955 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fdd54007490 con 0x7fdd600d3ea0 2024-01-19T04:56:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.407+0000 7fdd47fff640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd4c06aed0 0x7fdd4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.408+0000 7fdd47fff640 1 -- 172.21.15.31:0/619163955 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fdd54088c50 con 0x7fdd600d3ea0 2024-01-19T04:56:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.408+0000 7fdd5f7fe640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd4c06aed0 0x7fdd4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.408+0000 7fdd5f7fe640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd4c06aed0 0x7fdd4c06d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd50036000 tx=0x7fdd50008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.412+0000 7fdd47fff640 1 -- 172.21.15.31:0/619163955 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdd600c7610 con 0x7fdd600d3ea0 2024-01-19T04:56:36.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.605+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdd600c7610 con 0x7fdd600d3ea0 2024-01-19T04:56:36.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.606+0000 7fdd47fff640 1 -- 172.21.15.31:0/619163955 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd600c7610 con 0x7fdd600d3ea0 2024-01-19T04:56:36.605 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd4c06aed0 msgr2=0x7fdd4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd4c06aed0 0x7fdd4c06d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd50036000 tx=0x7fdd50008030 comp rx=0 tx=0).stop 2024-01-19T04:56:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd600d3ea0 msgr2=0x7fdd601588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd600d3ea0 0x7fdd601588b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c4d0 tx=0x7fdd5400c990 comp rx=0 tx=0).stop 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 shutdown_connections 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fdd4c06aed0 0x7fdd4c06d390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdd6015d720 0x7fdd600b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd600d3ea0 0x7fdd601588b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 --2- 172.21.15.31:0/619163955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd600d1c20 0x7fdd600c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 >> 172.21.15.31:0/619163955 conn(0x7fdd600d3310 msgr2=0x7fdd600d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 shutdown_connections 2024-01-19T04:56:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.608+0000 7fdd65ec2640 1 -- 172.21.15.31:0/619163955 wait complete. 2024-01-19T04:56:36.618 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:36.618 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:36.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.776+0000 7f671ed43640 1 Processor -- start 2024-01-19T04:56:36.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.776+0000 7f671ed43640 1 -- start start 2024-01-19T04:56:36.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.776+0000 7f671ed43640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f671815de70 0x7f671815e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f671ed43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67180b2d40 0x7f67180b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f671ed43640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67180cb8b0 con 0x7f67181580f0 2024-01-19T04:56:36.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f671ed43640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671806c300 con 0x7f671815de70 2024-01-19T04:56:36.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f671ed43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67180b36f0 con 0x7f67180b2d40 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f6717fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f671815de70 0x7f671815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f6717fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f671815de70 0x7f671815e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37326/0 (socket says 172.21.15.31:37326) 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f6717fff640 1 -- 172.21.15.31:0/4188231823 learned_addr learned my addr 172.21.15.31:0/4188231823 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f671d2e0640 1 --2- 172.21.15.31:0/4188231823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67180b2d40 0x7f67180b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f6717fff640 1 -- 172.21.15.31:0/4188231823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67180b2d40 msgr2=0x7f67180b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f6717fff640 1 --2- 172.21.15.31:0/4188231823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67180b2d40 0x7f67180b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f6717fff640 1 -- 172.21.15.31:0/4188231823 >> v1:172.21.15.137:6789/0 conn(0x7f67181580f0 legacy=0x7f671815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f6717fff640 1 -- 172.21.15.31:0/4188231823 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671815f740 con 0x7f671815de70 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.777+0000 7f671d2e0640 1 --2- 172.21.15.31:0/4188231823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67180b2d40 0x7f67180b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f6717fff640 1 --2- 172.21.15.31:0/4188231823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f671815de70 0x7f671815e320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f67040098a0 tx=0x7f670402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c4a4b0e3209fa4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f67177fe640 1 -- 172.21.15.31:0/4188231823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031640 con 0x7f671815de70 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f67177fe640 1 -- 172.21.15.31:0/4188231823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040317a0 con 0x7f671815de70 2024-01-19T04:56:36.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f67177fe640 1 -- 172.21.15.31:0/4188231823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031b30 con 0x7f671815de70 2024-01-19T04:56:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 -- 172.21.15.31:0/4188231823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f671815de70 msgr2=0x7f671815e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 --2- 172.21.15.31:0/4188231823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f671815de70 0x7f671815e320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f67040098a0 tx=0x7f670402b8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 -- 172.21.15.31:0/4188231823 shutdown_connections 2024-01-19T04:56:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 --2- 172.21.15.31:0/4188231823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67180b2d40 0x7f67180b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 --2- 172.21.15.31:0/4188231823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f671815de70 0x7f671815e320 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 -- 172.21.15.31:0/4188231823 >> 172.21.15.31:0/4188231823 conn(0x7f671806abe0 msgr2=0x7f6718066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 -- 172.21.15.31:0/4188231823 shutdown_connections 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.778+0000 7f671ed43640 1 -- 172.21.15.31:0/4188231823 wait complete. 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.779+0000 7f671ed43640 1 Processor -- start 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.780+0000 7f671ed43640 1 -- start start 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.780+0000 7f671ed43640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67180b2d40 0x7f67180c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.780+0000 7f671ed43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67181580f0 0x7f67180d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.780+0000 7f671ed43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f671815de70 0x7f67181613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.780+0000 7f671ed43640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6718158530 con 0x7f67180b2d40 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.780+0000 7f671ed43640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180d16b0 con 0x7f671815de70 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.780+0000 7f671ed43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67180d1990 con 0x7f67181580f0 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67180b2d40 0x7f67180c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67180b2d40 0x7f67180c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37340/0 (socket says 172.21.15.31:37340) 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 -- 172.21.15.31:0/2148694394 learned_addr learned my addr 172.21.15.31:0/2148694394 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 -- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67181580f0 msgr2=0x7f67180d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67181580f0 0x7f67180d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671d2e0640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f671815de70 0x7f67181613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 -- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f671815de70 msgr2=0x7f67181613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f671815de70 0x7f67181613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 -- 172.21.15.31:0/2148694394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040094d0 con 0x7f67180b2d40 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671d2e0640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f671815de70 0x7f67181613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671cadf640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67180b2d40 0x7f67180c6fe0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f670c00c430 tx=0x7f670c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f67157fa640 1 -- 172.21.15.31:0/2148694394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c011790 con 0x7f67180b2d40 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f67157fa640 1 -- 172.21.15.31:0/2148694394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c0118f0 con 0x7f67180b2d40 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67180d1c70 con 0x7f67180b2d40 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f67157fa640 1 -- 172.21.15.31:0/2148694394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c011ce0 con 0x7f67180b2d40 2024-01-19T04:56:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.781+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67180c7520 con 0x7f67180b2d40 2024-01-19T04:56:36.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.782+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6718160da0 con 0x7f67180b2d40 2024-01-19T04:56:36.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.783+0000 7f67157fa640 1 -- 172.21.15.31:0/2148694394 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f670c01c460 con 0x7f67180b2d40 2024-01-19T04:56:36.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.783+0000 7f67157fa640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f66fc06aed0 0x7f66fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:36.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.783+0000 7f67157fa640 1 -- 172.21.15.31:0/2148694394 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f670c0897a0 con 0x7f67180b2d40 2024-01-19T04:56:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.783+0000 7f6717fff640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f66fc06aed0 0x7f66fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.784+0000 7f6717fff640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f66fc06aed0 0x7f66fc06d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6704036000 tx=0x7f6704030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.787+0000 7f67157fa640 1 -- 172.21.15.31:0/2148694394 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f670c0544e0 con 0x7f67180b2d40 2024-01-19T04:56:36.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.964+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67180696e0 con 0x7f67180b2d40 2024-01-19T04:56:36.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.965+0000 7f67157fa640 1 -- 172.21.15.31:0/2148694394 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f670c057b00 con 0x7f67180b2d40 2024-01-19T04:56:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:36.966 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f66fc06aed0 msgr2=0x7f66fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f66fc06aed0 0x7f66fc06d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6704036000 tx=0x7f6704030000 comp rx=0 tx=0).stop 2024-01-19T04:56:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67180b2d40 msgr2=0x7f67180c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67180b2d40 0x7f67180c6fe0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f670c00c430 tx=0x7f670c00c8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 shutdown_connections 2024-01-19T04:56:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f66fc06aed0 0x7f66fc06d390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f671815de70 0x7f67181613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67181580f0 0x7f67180d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 --2- 172.21.15.31:0/2148694394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67180b2d40 0x7f67180c6fe0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:36.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 >> 172.21.15.31:0/2148694394 conn(0x7f671806abe0 msgr2=0x7f67180b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:36.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 shutdown_connections 2024-01-19T04:56:36.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:36.967+0000 7f671ed43640 1 -- 172.21.15.31:0/2148694394 wait complete. 2024-01-19T04:56:36.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:56:23.785927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:56:24.240600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-01-19T04:56:32.396495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:36.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:37.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.082+0000 7f7437d05640 1 Processor -- start 2024-01-19T04:56:37.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.083+0000 7f7437d05640 1 -- start start 2024-01-19T04:56:37.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.083+0000 7f7437d05640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f743015d720 0x7f74300d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.083+0000 7f7437d05640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74300d1c20 0x7f74300b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.083+0000 7f7437d05640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74300cf160 con 0x7f74300d3ea0 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.083+0000 7f7437d05640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74300ce8c0 con 0x7f74300d1c20 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.083+0000 7f7437d05640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74300ceba0 con 0x7f743015d720 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.083+0000 7f74362a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74300d1c20 0x7f74300b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f74362a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74300d1c20 0x7f74300b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37350/0 (socket says 172.21.15.31:37350) 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f74362a2640 1 -- 172.21.15.31:0/878027355 learned_addr learned my addr 172.21.15.31:0/878027355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f7435aa1640 1 --2- 172.21.15.31:0/878027355 >> v2:172.21.15.137:3300/0 conn(0x7f743015d720 0x7f74300d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f74362a2640 1 -- 172.21.15.31:0/878027355 >> v2:172.21.15.137:3300/0 conn(0x7f743015d720 msgr2=0x7f74300d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f74362a2640 1 --2- 172.21.15.31:0/878027355 >> v2:172.21.15.137:3300/0 conn(0x7f743015d720 0x7f74300d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f74362a2640 1 -- 172.21.15.31:0/878027355 >> v1:172.21.15.137:6789/0 conn(0x7f74300d3ea0 legacy=0x7f74300d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f74362a2640 1 -- 172.21.15.31:0/878027355 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74300cee80 con 0x7f74300d1c20 2024-01-19T04:56:37.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.084+0000 7f7435aa1640 1 --2- 172.21.15.31:0/878027355 >> v2:172.21.15.137:3300/0 conn(0x7f743015d720 0x7f74300d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:37.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f74362a2640 1 --2- 172.21.15.31:0/878027355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300b3000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7430067310 tx=0x7f742802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24087f16c3a5248 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7434a9f640 1 -- 172.21.15.31:0/878027355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742802c960 con 0x7f74300d1c20 2024-01-19T04:56:37.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7434a9f640 1 -- 172.21.15.31:0/878027355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742802cac0 con 0x7f74300d1c20 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7434a9f640 1 -- 172.21.15.31:0/878027355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742802ce50 con 0x7f74300d1c20 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 -- 172.21.15.31:0/878027355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 msgr2=0x7f74300b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 --2- 172.21.15.31:0/878027355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300b3000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7430067310 tx=0x7f742802bdc0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 -- 172.21.15.31:0/878027355 shutdown_connections 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 --2- 172.21.15.31:0/878027355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300b3000 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 --2- 172.21.15.31:0/878027355 >> v2:172.21.15.137:3300/0 conn(0x7f743015d720 0x7f74300d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 -- 172.21.15.31:0/878027355 >> 172.21.15.31:0/878027355 conn(0x7f74300d3310 msgr2=0x7f74300d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 -- 172.21.15.31:0/878027355 shutdown_connections 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.085+0000 7f7437d05640 1 -- 172.21.15.31:0/878027355 wait complete. 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.086+0000 7f7437d05640 1 Processor -- start 2024-01-19T04:56:37.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7437d05640 1 -- start start 2024-01-19T04:56:37.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7437d05640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7437d05640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74300d3ea0 0x7f7430159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7437d05640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f743015d720 0x7f743015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7437d05640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74300c7e90 con 0x7f74300d1c20 2024-01-19T04:56:37.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7437d05640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74301505d0 con 0x7f743015d720 2024-01-19T04:56:37.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7437d05640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74300c80e0 con 0x7f74300d3ea0 2024-01-19T04:56:37.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7435aa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7435aa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37362/0 (socket says 172.21.15.31:37362) 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.087+0000 7f7435aa1640 1 -- 172.21.15.31:0/4111947149 learned_addr learned my addr 172.21.15.31:0/4111947149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7435aa1640 1 -- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74300d3ea0 msgr2=0x7f7430159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7435aa1640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74300d3ea0 0x7f7430159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7435aa1640 1 -- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f743015d720 msgr2=0x7f743015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7435aa1640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f743015d720 0x7f743015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7435aa1640 1 -- 172.21.15.31:0/4111947149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428009b30 con 0x7f74300d1c20 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7435aa1640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300c9180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c7f0 tx=0x7f742c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f74267fc640 1 -- 172.21.15.31:0/4111947149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c007820 con 0x7f74300d1c20 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f74267fc640 1 -- 172.21.15.31:0/4111947149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c007980 con 0x7f74300d1c20 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74300c83c0 con 0x7f74300d1c20 2024-01-19T04:56:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f74267fc640 1 -- 172.21.15.31:0/4111947149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c007cf0 con 0x7f74300d1c20 2024-01-19T04:56:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.088+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74300c8980 con 0x7f74300d1c20 2024-01-19T04:56:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.089+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7430074900 con 0x7f74300d1c20 2024-01-19T04:56:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.089+0000 7f74267fc640 1 -- 172.21.15.31:0/4111947149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f742c0113f0 con 0x7f74300d1c20 2024-01-19T04:56:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.090+0000 7f74267fc640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f741406aed0 0x7f741406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.090+0000 7f74267fc640 1 -- 172.21.15.31:0/4111947149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f742c089a20 con 0x7f74300d1c20 2024-01-19T04:56:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.090+0000 7f74352a0640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f741406aed0 0x7f741406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.090+0000 7f74352a0640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f741406aed0 0x7f741406d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f74300c9b80 tx=0x7f74200079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.094+0000 7f74267fc640 1 -- 172.21.15.31:0/4111947149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f742c0547f0 con 0x7f74300d1c20 2024-01-19T04:56:37.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.269+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74300d1840 con 0x7f74300d1c20 2024-01-19T04:56:37.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.269+0000 7f74267fc640 1 -- 172.21.15.31:0/4111947149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f742c01f0a0 con 0x7f74300d1c20 2024-01-19T04:56:37.269 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f741406aed0 msgr2=0x7f741406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f741406aed0 0x7f741406d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f74300c9b80 tx=0x7f74200079b0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 msgr2=0x7f74300c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300c9180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c7f0 tx=0x7f742c00ccb0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 shutdown_connections 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f741406aed0 0x7f741406d390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f743015d720 0x7f743015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74300d3ea0 0x7f7430159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 --2- 172.21.15.31:0/4111947149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74300d1c20 0x7f74300c9180 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 >> 172.21.15.31:0/4111947149 conn(0x7f74300d3310 msgr2=0x7f74300cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 shutdown_connections 2024-01-19T04:56:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.271+0000 7f7437d05640 1 -- 172.21.15.31:0/4111947149 wait complete. 2024-01-19T04:56:37.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:37.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.386+0000 7f9d3e9a4640 1 Processor -- start 2024-01-19T04:56:37.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.386+0000 7f9d3e9a4640 1 -- start start 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d3e9a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 0x7f9d380d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d3e9a4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9d380d1fe0 0x7f9d380b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d3e9a4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380cef10 con 0x7f9d3815d700 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d3e9a4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d380ce300 con 0x7f9d380d1fe0 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d3e9a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380ce5e0 con 0x7f9d380d11b0 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d37fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9d3815d700 0x7f9d380d18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58998/0 (socket says 172.21.15.31:58998) 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d37fff640 1 -- 172.21.15.31:0/3418463177 learned_addr learned my addr 172.21.15.31:0/3418463177 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d377fe640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 0x7f9d380d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/3418463177 >> v2:172.21.15.137:3300/0 conn(0x7f9d380d1fe0 0x7f9d380b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233730452 0 0) 0x7f9d380cef10 con 0x7f9d3815d700 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d24003520 con 0x7f9d3815d700 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 124620265 0 0) 0x7f9d24003520 con 0x7f9d3815d700 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 msgr2=0x7f9d380d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 0x7f9d380d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 >> v2:172.21.15.137:3300/0 conn(0x7f9d380d1fe0 msgr2=0x7f9d380b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 --2- 172.21.15.31:0/3418463177 >> v2:172.21.15.137:3300/0 conn(0x7f9d380d1fe0 0x7f9d380b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380ce8c0 con 0x7f9d3815d700 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f9d28001670 con 0x7f9d3815d700 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 >> v1:172.21.15.31:6789/0 conn(0x7f9d3815d700 legacy=0x7f9d380d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24003970 0x7f9d24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 0x7f9d240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380ce300 con 0x7f9d24004d00 2024-01-19T04:56:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380ce5e0 con 0x7f9d24003970 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.387+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2400b980 con 0x7f9d380d11b0 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d37fff640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24003970 0x7f9d24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d377fe640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 0x7f9d240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d3cf41640 1 -- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 msgr2=0x7f9d240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 0x7f9d240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d3cf41640 1 -- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24003970 msgr2=0x7f9d24003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24003970 0x7f9d24003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d3cf41640 1 -- 172.21.15.31:0/3418463177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24003520 con 0x7f9d24004d00 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.388+0000 7f9d377fe640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d380d11b0 0x7f9d240047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009fc0 tx=0x7f9d240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3698cf95fe21fd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3002cc60 con 0x7f9d24004d00 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3000f040 con 0x7f9d24004d00 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d36ffd640 1 -- 172.21.15.31:0/3418463177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30034440 con 0x7f9d24004d00 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/3418463177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d24004d00 msgr2=0x7f9d24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3e9a4640 1 --2- 172.21.15.31:0/3418463177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009fc0 tx=0x7f9d240057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3cf41640 1 -- 172.21.15.31:0/3418463177 reap_dead start 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/3418463177 shutdown_connections 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/3418463177 >> 172.21.15.31:0/3418463177 conn(0x7f9d380d32f0 msgr2=0x7f9d380d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/3418463177 shutdown_connections 2024-01-19T04:56:37.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.389+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/3418463177 wait complete. 2024-01-19T04:56:37.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.390+0000 7f9d3e9a4640 1 Processor -- start 2024-01-19T04:56:37.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3e9a4640 1 -- start start 2024-01-19T04:56:37.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3e9a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24004d00 0x7f9d380c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3e9a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d381f3080 0x7f9d38160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3e9a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d381f5d50 0x7f9d3815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3e9a4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380c8300 con 0x7f9d381f5d50 2024-01-19T04:56:37.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3e9a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38160c40 con 0x7f9d24004d00 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3e9a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380c8540 con 0x7f9d381f3080 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3cf41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d381f5d50 0x7f9d3815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3cf41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d381f5d50 0x7f9d3815e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37388/0 (socket says 172.21.15.31:37388) 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.391+0000 7f9d3cf41640 1 -- 172.21.15.31:0/2580583940 learned_addr learned my addr 172.21.15.31:0/2580583940 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d37fff640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24004d00 0x7f9d380c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d377fe640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d381f3080 0x7f9d38160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3cf41640 1 -- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d381f3080 msgr2=0x7f9d38160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d381f3080 0x7f9d38160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3cf41640 1 -- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24004d00 msgr2=0x7f9d380c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24004d00 0x7f9d380c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3cf41640 1 -- 172.21.15.31:0/2580583940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380c7360 con 0x7f9d381f5d50 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d37fff640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24004d00 0x7f9d380c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:37.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3cf41640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d381f5d50 0x7f9d3815e7f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d30037040 tx=0x7f9d30009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d34ff9640 1 -- 172.21.15.31:0/2580583940 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30043070 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d34ff9640 1 -- 172.21.15.31:0/2580583940 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30034b50 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d380c7640 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d380c7c00 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.392+0000 7f9d34ff9640 1 -- 172.21.15.31:0/2580583940 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30003990 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.393+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c005290 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.394+0000 7f9d34ff9640 1 -- 172.21.15.31:0/2580583940 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9d3003e440 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.394+0000 7f9d34ff9640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1806aed0 0x7f9d1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.394+0000 7f9d34ff9640 1 -- 172.21.15.31:0/2580583940 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9d300ab0b0 con 0x7f9d381f5d50 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.395+0000 7f9d37fff640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1806aed0 0x7f9d1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.395+0000 7f9d37fff640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1806aed0 0x7f9d1806d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d28000c00 tx=0x7f9d28009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.399+0000 7f9d34ff9640 1 -- 172.21.15.31:0/2580583940 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9d30075e80 con 0x7f9d381f5d50 2024-01-19T04:56:37.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.574+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d1c005810 con 0x7f9d381f5d50 2024-01-19T04:56:37.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.575+0000 7f9d34ff9640 1 -- 172.21.15.31:0/2580583940 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d300794a0 con 0x7f9d381f5d50 2024-01-19T04:56:37.574 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:37.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1806aed0 msgr2=0x7f9d1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1806aed0 0x7f9d1806d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d28000c00 tx=0x7f9d28009040 comp rx=0 tx=0).stop 2024-01-19T04:56:37.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d381f5d50 msgr2=0x7f9d3815e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d381f5d50 0x7f9d3815e7f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d30037040 tx=0x7f9d30009750 comp rx=0 tx=0).stop 2024-01-19T04:56:37.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 shutdown_connections 2024-01-19T04:56:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9d1806aed0 0x7f9d1806d390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d381f5d50 0x7f9d3815e7f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9d381f3080 0x7f9d38160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 --2- 172.21.15.31:0/2580583940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d24004d00 0x7f9d380c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 >> 172.21.15.31:0/2580583940 conn(0x7f9d380d32f0 msgr2=0x7f9d3815c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 shutdown_connections 2024-01-19T04:56:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.577+0000 7f9d3e9a4640 1 -- 172.21.15.31:0/2580583940 wait complete. 2024-01-19T04:56:37.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:37.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.691+0000 7f95d163a640 1 Processor -- start 2024-01-19T04:56:37.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.691+0000 7f95d163a640 1 -- start start 2024-01-19T04:56:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95d163a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f95cc0d1c20 0x7f95cc0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95d163a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc0cf160 con 0x7f95cc0d3ea0 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95d163a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc0ce8c0 con 0x7f95cc15d720 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95d163a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0ceba0 con 0x7f95cc0d1c20 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95cb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95cc0d3ea0 0x7f95cc0b3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59004/0 (socket says 172.21.15.31:59004) 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95cb7fe640 1 -- 172.21.15.31:0/2466382643 learned_addr learned my addr 172.21.15.31:0/2466382643 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95ca7fc640 1 --2- 172.21.15.31:0/2466382643 >> v2:172.21.15.137:3300/0 conn(0x7f95cc0d1c20 0x7f95cc0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525934408 0 0) 0x7f95cc0cf160 con 0x7f95cc0d3ea0 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.692+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b8003520 con 0x7f95cc0d3ea0 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2210737267 0 0) 0x7f95b8003520 con 0x7f95cc0d3ea0 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 >> v2:172.21.15.137:3300/0 conn(0x7f95cc0d1c20 msgr2=0x7f95cc0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 --2- 172.21.15.31:0/2466382643 >> v2:172.21.15.137:3300/0 conn(0x7f95cc0d1c20 0x7f95cc0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 >> v1:172.21.15.137:6789/0 conn(0x7f95cc15d720 legacy=0x7f95cc0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0cee80 con 0x7f95cc0d3ea0 2024-01-19T04:56:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f95bc0019f0 con 0x7f95cc0d3ea0 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 >> v1:172.21.15.31:6789/0 conn(0x7f95cc0d3ea0 legacy=0x7f95cc0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95b8003970 0x7f95b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80052a0 0x7f95b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0ceba0 con 0x7f95b80047c0 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b800c170 con 0x7f95b8003970 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b800c330 con 0x7f95b80052a0 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95ca7fc640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95caffd640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95b8003970 0x7f95b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95cb7fe640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80052a0 0x7f95b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95ca7fc640 1 -- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80052a0 msgr2=0x7f95b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95ca7fc640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80052a0 0x7f95b800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95ca7fc640 1 -- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95b8003970 msgr2=0x7f95b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:37.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95ca7fc640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95b8003970 0x7f95b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95ca7fc640 1 -- 172.21.15.31:0/2466382643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b8003520 con 0x7f95b80047c0 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95caffd640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95b8003970 0x7f95b8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.693+0000 7f95cb7fe640 1 -- 172.21.15.31:0/2466382643 reap_dead start 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.694+0000 7f95ca7fc640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009a90 tx=0x7f95b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78463ae4b49e560 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.694+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b402c680 con 0x7f95b80047c0 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.694+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b402c7e0 con 0x7f95b80047c0 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.694+0000 7f95c9ffb640 1 -- 172.21.15.31:0/2466382643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b402cb70 con 0x7f95b80047c0 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.695+0000 7f95d163a640 1 -- 172.21.15.31:0/2466382643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95b80047c0 msgr2=0x7f95b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.695+0000 7f95d163a640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009a90 tx=0x7f95b80057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.695+0000 7f95d163a640 1 -- 172.21.15.31:0/2466382643 shutdown_connections 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.695+0000 7f95d163a640 1 --2- 172.21.15.31:0/2466382643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.695+0000 7f95d163a640 1 -- 172.21.15.31:0/2466382643 >> 172.21.15.31:0/2466382643 conn(0x7f95cc0d3310 msgr2=0x7f95cc0d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.695+0000 7f95d163a640 1 -- 172.21.15.31:0/2466382643 shutdown_connections 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.695+0000 7f95d163a640 1 -- 172.21.15.31:0/2466382643 wait complete. 2024-01-19T04:56:37.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.696+0000 7f95d163a640 1 Processor -- start 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.696+0000 7f95d163a640 1 -- start start 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95d163a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80047c0 0x7f95cc1603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95d163a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95cc1608f0 0x7f95cc0ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95d163a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95cc0ca920 0x7f95cc151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95d163a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0cada0 con 0x7f95cc0ca920 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95d163a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0b88f0 con 0x7f95cc1608f0 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95d163a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc15f5f0 con 0x7f95b80047c0 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95cc0ca920 0x7f95cc151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95cc0ca920 0x7f95cc151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37392/0 (socket says 172.21.15.31:37392) 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 -- 172.21.15.31:0/614551374 learned_addr learned my addr 172.21.15.31:0/614551374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95caffd640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80047c0 0x7f95cc1603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95ca7fc640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95cc1608f0 0x7f95cc0ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 -- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80047c0 msgr2=0x7f95cc1603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80047c0 0x7f95cc1603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 -- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95cc1608f0 msgr2=0x7f95cc0ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95cc1608f0 0x7f95cc0ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 -- 172.21.15.31:0/614551374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc15f8d0 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95ca7fc640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95cc1608f0 0x7f95cc0ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95cb7fe640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95cc0ca920 0x7f95cc151690 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95cc08c490 tx=0x7f95bc007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95c37fe640 1 -- 172.21.15.31:0/614551374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00ea80 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95c37fe640 1 -- 172.21.15.31:0/614551374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc00ebe0 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc15fbb0 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.697+0000 7f95c37fe640 1 -- 172.21.15.31:0/614551374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc013420 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.698+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc151bd0 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.698+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc1600d0 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.699+0000 7f95c37fe640 1 -- 172.21.15.31:0/614551374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f95bc013580 con 0x7f95cc0ca920 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.699+0000 7f95c37fe640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95ac06aed0 0x7f95ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.699+0000 7f95c37fe640 1 -- 172.21.15.31:0/614551374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f95bc089a90 con 0x7f95cc0ca920 2024-01-19T04:56:37.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.700+0000 7f95caffd640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95ac06aed0 0x7f95ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:37.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.700+0000 7f95caffd640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95ac06aed0 0x7f95ac06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:37.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.704+0000 7f95c37fe640 1 -- 172.21.15.31:0/614551374 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f95cc1600d0 con 0x7f95cc0ca920 2024-01-19T04:56:37.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.882+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f95cc1600d0 con 0x7f95cc0ca920 2024-01-19T04:56:37.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.882+0000 7f95c37fe640 1 -- 172.21.15.31:0/614551374 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95cc1600d0 con 0x7f95cc0ca920 2024-01-19T04:56:37.882 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95ac06aed0 msgr2=0x7f95ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95ac06aed0 0x7f95ac06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4009040 comp rx=0 tx=0).stop 2024-01-19T04:56:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95cc0ca920 msgr2=0x7f95cc151690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95cc0ca920 0x7f95cc151690 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95cc08c490 tx=0x7f95bc007bc0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 shutdown_connections 2024-01-19T04:56:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95ac06aed0 0x7f95ac06d390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95cc0ca920 0x7f95cc151690 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95cc1608f0 0x7f95cc0ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 --2- 172.21.15.31:0/614551374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f95b80047c0 0x7f95cc1603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 >> 172.21.15.31:0/614551374 conn(0x7f95cc0d3310 msgr2=0x7f95cc15c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 shutdown_connections 2024-01-19T04:56:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.884+0000 7f95d163a640 1 -- 172.21.15.31:0/614551374 wait complete. 2024-01-19T04:56:37.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-19T04:56:37.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.999+0000 7faf74341640 1 Processor -- start 2024-01-19T04:56:37.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:37.999+0000 7faf74341640 1 -- start start 2024-01-19T04:56:38.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.000+0000 7faf74341640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf6c15b330 0x7faf6c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.000+0000 7faf74341640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 0x7faf6c1614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.000+0000 7faf74341640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6c0b2580 con 0x7faf6c161a00 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.000+0000 7faf74341640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6c15e8b0 con 0x7faf6c15b330 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.000+0000 7faf74341640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c15df10 con 0x7faf6c0b2120 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf6c15b330 0x7faf6c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf6c15b330 0x7faf6c0b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37400/0 (socket says 172.21.15.31:37400) 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 -- 172.21.15.31:0/353546538 learned_addr learned my addr 172.21.15.31:0/353546538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 -- 172.21.15.31:0/353546538 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 msgr2=0x7faf6c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 --2- 172.21.15.31:0/353546538 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 0x7faf6c1614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 -- 172.21.15.31:0/353546538 >> v1:172.21.15.137:6789/0 conn(0x7faf6c161a00 legacy=0x7faf6c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 -- 172.21.15.31:0/353546538 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c15e1f0 con 0x7faf6c15b330 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.001+0000 7faf720dd640 1 --2- 172.21.15.31:0/353546538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b1be0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf68009870 tx=0x7faf6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722a06c881a04cce server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf710db640 1 -- 172.21.15.31:0/353546538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68031640 con 0x7faf6c15b330 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf710db640 1 -- 172.21.15.31:0/353546538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf680317a0 con 0x7faf6c15b330 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf710db640 1 -- 172.21.15.31:0/353546538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68031b30 con 0x7faf6c15b330 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 -- 172.21.15.31:0/353546538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 msgr2=0x7faf6c0b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 --2- 172.21.15.31:0/353546538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b1be0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf68009870 tx=0x7faf6802b8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 -- 172.21.15.31:0/353546538 shutdown_connections 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 --2- 172.21.15.31:0/353546538 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 0x7faf6c1614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 --2- 172.21.15.31:0/353546538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b1be0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 -- 172.21.15.31:0/353546538 >> 172.21.15.31:0/353546538 conn(0x7faf6c0d0000 msgr2=0x7faf6c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:38.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 -- 172.21.15.31:0/353546538 shutdown_connections 2024-01-19T04:56:38.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.002+0000 7faf74341640 1 -- 172.21.15.31:0/353546538 wait complete. 2024-01-19T04:56:38.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.003+0000 7faf74341640 1 Processor -- start 2024-01-19T04:56:38.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf74341640 1 -- start start 2024-01-19T04:56:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf74341640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 0x7faf6c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf74341640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf74341640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf6c161a00 0x7faf6c15ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf74341640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c1e7f60 con 0x7faf6c15b330 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf74341640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0b52b0 con 0x7faf6c161a00 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf74341640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c1e72e0 con 0x7faf6c0b2120 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf718dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf718dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37402/0 (socket says 172.21.15.31:37402) 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.004+0000 7faf718dc640 1 -- 172.21.15.31:0/2463433523 learned_addr learned my addr 172.21.15.31:0/2463433523 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf728de640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf6c161a00 0x7faf6c15ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf720dd640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 0x7faf6c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf718dc640 1 -- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 msgr2=0x7faf6c0c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf718dc640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 0x7faf6c0c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf718dc640 1 -- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf6c161a00 msgr2=0x7faf6c15ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf718dc640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf6c161a00 0x7faf6c15ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf718dc640 1 -- 172.21.15.31:0/2463433523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf680094d0 con 0x7faf6c15b330 2024-01-19T04:56:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf728de640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf6c161a00 0x7faf6c15ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf718dc640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b2e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00b4a0 tx=0x7faf5c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf62ffd640 1 -- 172.21.15.31:0/2463433523 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c010610 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf62ffd640 1 -- 172.21.15.31:0/2463433523 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c010770 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c1e75c0 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf62ffd640 1 -- 172.21.15.31:0/2463433523 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c010b20 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.005+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c1e7b80 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.006+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c1e80a0 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.007+0000 7faf62ffd640 1 -- 172.21.15.31:0/2463433523 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7faf5c010c80 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.007+0000 7faf62ffd640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faf5006aed0 0x7faf5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.007+0000 7faf62ffd640 1 -- 172.21.15.31:0/2463433523 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7faf5c089360 con 0x7faf6c15b330 2024-01-19T04:56:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.007+0000 7faf720dd640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faf5006aed0 0x7faf5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.008+0000 7faf720dd640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faf5006aed0 0x7faf5006d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf68036000 tx=0x7faf6803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:38.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.011+0000 7faf62ffd640 1 -- 172.21.15.31:0/2463433523 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7faf6c1e80a0 con 0x7faf6c15b330 2024-01-19T04:56:38.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.189+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7faf6c1e80a0 con 0x7faf6c15b330 2024-01-19T04:56:38.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.189+0000 7faf62ffd640 1 -- 172.21.15.31:0/2463433523 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7faf6c1e80a0 con 0x7faf6c15b330 2024-01-19T04:56:38.189 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faf5006aed0 msgr2=0x7faf5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faf5006aed0 0x7faf5006d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf68036000 tx=0x7faf6803c040 comp rx=0 tx=0).stop 2024-01-19T04:56:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 msgr2=0x7faf6c0b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b2e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00b4a0 tx=0x7faf5c00b960 comp rx=0 tx=0).stop 2024-01-19T04:56:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 shutdown_connections 2024-01-19T04:56:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faf5006aed0 0x7faf5006d390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf6c161a00 0x7faf6c15ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c15b330 0x7faf6c0b2e70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 --2- 172.21.15.31:0/2463433523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faf6c0b2120 0x7faf6c0c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 >> 172.21.15.31:0/2463433523 conn(0x7faf6c0d0000 msgr2=0x7faf6c15b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 shutdown_connections 2024-01-19T04:56:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.191+0000 7faf74341640 1 -- 172.21.15.31:0/2463433523 wait complete. 2024-01-19T04:56:38.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:38.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.304+0000 7f35c7bc9640 1 Processor -- start 2024-01-19T04:56:38.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.304+0000 7f35c7bc9640 1 -- start start 2024-01-19T04:56:38.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c7bc9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c00cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c7bc9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35c006acc0 0x7f35c0071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c7bc9640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c00c6130 con 0x7f35c00cc490 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c7bc9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c00cc910 con 0x7f35c006acc0 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c7bc9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c00ce630 con 0x7f35c015d340 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c6166640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35c006acc0 0x7f35c0071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c6166640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35c006acc0 0x7f35c0071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37414/0 (socket says 172.21.15.31:37414) 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c6166640 1 -- 172.21.15.31:0/1555661165 learned_addr learned my addr 172.21.15.31:0/1555661165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c5965640 1 --2- 172.21.15.31:0/1555661165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c00cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c6166640 1 -- 172.21.15.31:0/1555661165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 msgr2=0x7f35c00cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c6166640 1 --2- 172.21.15.31:0/1555661165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c00cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c6166640 1 -- 172.21.15.31:0/1555661165 >> v1:172.21.15.137:6789/0 conn(0x7f35c00cc490 legacy=0x7f35c006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.305+0000 7f35c6166640 1 -- 172.21.15.31:0/1555661165 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c00c5520 con 0x7f35c006acc0 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.306+0000 7f35c6166640 1 --2- 172.21.15.31:0/1555661165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c006acc0 0x7f35c0071340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009e50 tx=0x7f35b802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fc2aa642b7e2e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.306+0000 7f35c4963640 1 -- 172.21.15.31:0/1555661165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b802c9e0 con 0x7f35c006acc0 2024-01-19T04:56:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.306+0000 7f35c4963640 1 -- 172.21.15.31:0/1555661165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b802cb40 con 0x7f35c006acc0 2024-01-19T04:56:38.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.306+0000 7f35c4963640 1 -- 172.21.15.31:0/1555661165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b80343f0 con 0x7f35c006acc0 2024-01-19T04:56:38.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1555661165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c006acc0 msgr2=0x7f35c0071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1555661165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c006acc0 0x7f35c0071340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009e50 tx=0x7f35b802be40 comp rx=0 tx=0).stop 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1555661165 shutdown_connections 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1555661165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c006acc0 0x7f35c0071340 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1555661165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c00cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1555661165 >> 172.21.15.31:0/1555661165 conn(0x7f35c00b1c50 msgr2=0x7f35c00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1555661165 shutdown_connections 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.307+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1555661165 wait complete. 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.308+0000 7f35c7bc9640 1 Processor -- start 2024-01-19T04:56:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.308+0000 7f35c7bc9640 1 -- start start 2024-01-19T04:56:38.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c7bc9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35c006acc0 0x7f35c0160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c7bc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c00cc490 0x7f35c0156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c7bc9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c7bc9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01570b0 con 0x7f35c00cc490 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c7bc9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c00add80 con 0x7f35c006acc0 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c7bc9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c015d720 con 0x7f35c015d340 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c00cc490 0x7f35c0156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c00cc490 0x7f35c0156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37430/0 (socket says 172.21.15.31:37430) 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 -- 172.21.15.31:0/1061860243 learned_addr learned my addr 172.21.15.31:0/1061860243 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5965640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35c006acc0 0x7f35c0160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c6166640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 -- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 msgr2=0x7f35c015b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c015b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 -- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35c006acc0 msgr2=0x7f35c0160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35c006acc0 0x7f35c0160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 -- 172.21.15.31:0/1061860243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b8009b30 con 0x7f35c00cc490 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5965640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35c006acc0 0x7f35c0160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.309+0000 7f35c5164640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c00cc490 0x7f35c0156b70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f35b000c450 tx=0x7f35b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.310+0000 7f35b67fc640 1 -- 172.21.15.31:0/1061860243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00052d0 con 0x7f35c00cc490 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.310+0000 7f35b67fc640 1 -- 172.21.15.31:0/1061860243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b0005430 con 0x7f35c00cc490 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.310+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c015d9c0 con 0x7f35c00cc490 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.310+0000 7f35b67fc640 1 -- 172.21.15.31:0/1061860243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0012870 con 0x7f35c00cc490 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.310+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c015df80 con 0x7f35c00cc490 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.310+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c0074900 con 0x7f35c00cc490 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.311+0000 7f35b67fc640 1 -- 172.21.15.31:0/1061860243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f35b00129d0 con 0x7f35c00cc490 2024-01-19T04:56:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.311+0000 7f35b67fc640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35a406aed0 0x7f35a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.311+0000 7f35b67fc640 1 -- 172.21.15.31:0/1061860243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f35b0089220 con 0x7f35c00cc490 2024-01-19T04:56:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.312+0000 7f35c5965640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35a406aed0 0x7f35a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.312+0000 7f35c5965640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35a406aed0 0x7f35a406d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f35bc006fd0 tx=0x7f35bc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.316+0000 7f35b67fc640 1 -- 172.21.15.31:0/1061860243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f35b0053ff0 con 0x7f35c00cc490 2024-01-19T04:56:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.506+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35c0073810 con 0x7f35c00cc490 2024-01-19T04:56:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.506+0000 7f35b67fc640 1 -- 172.21.15.31:0/1061860243 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f35b0057610 con 0x7f35c00cc490 2024-01-19T04:56:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35a406aed0 msgr2=0x7f35a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35a406aed0 0x7f35a406d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f35bc006fd0 tx=0x7f35bc008040 comp rx=0 tx=0).stop 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c00cc490 msgr2=0x7f35c0156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c00cc490 0x7f35c0156b70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f35b000c450 tx=0x7f35b000c910 comp rx=0 tx=0).stop 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 shutdown_connections 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f35a406aed0 0x7f35a406d390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f35c015d340 0x7f35c015b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35c00cc490 0x7f35c0156b70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 --2- 172.21.15.31:0/1061860243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35c006acc0 0x7f35c0160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 >> 172.21.15.31:0/1061860243 conn(0x7f35c00b1c50 msgr2=0x7f35c00cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:38.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 shutdown_connections 2024-01-19T04:56:38.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:38.508+0000 7f35c7bc9640 1 -- 172.21.15.31:0/1061860243 wait complete. 2024-01-19T04:56:38.518 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:56:38.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:38.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:38.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[50945]: fuse finished with error 0 and tester_r 0 2024-01-19T04:56:38.546 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:43.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:56:43.706 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:56:44.647 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:56:44.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:44.647 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:56:44.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:56:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.818+0000 7fb8293f1640 1 Processor -- start 2024-01-19T04:56:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.818+0000 7fb8293f1640 1 -- start start 2024-01-19T04:56:44.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8293f1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8240b3030 0x7fb824160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:44.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8293f1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb824071df0 0x7fb8241614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8293f1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8240b3510 con 0x7fb82415d300 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8293f1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8240b37f0 con 0x7fb8240b3030 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8293f1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240b3ad0 con 0x7fb824071df0 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb81bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8240b3030 0x7fb824160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb822ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb82415d300 0x7fb82406ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58516/0 (socket says 172.21.15.31:58516) 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb822ffd640 1 -- 172.21.15.31:0/2855403432 learned_addr learned my addr 172.21.15.31:0/2855403432 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8227fc640 1 -- 172.21.15.31:0/2855403432 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244234075 0 0) 0x7fb8240b3510 con 0x7fb82415d300 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8237fe640 1 --2- 172.21.15.31:0/2855403432 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb824071df0 0x7fb8241614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb8227fc640 1 -- 172.21.15.31:0/2855403432 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb810003520 con 0x7fb82415d300 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb81bfff640 1 -- 172.21.15.31:0/2855403432 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb824071df0 msgr2=0x7fb8241614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb81bfff640 1 --2- 172.21.15.31:0/2855403432 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb824071df0 0x7fb8241614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb81bfff640 1 -- 172.21.15.31:0/2855403432 >> v1:172.21.15.31:6789/0 conn(0x7fb82415d300 legacy=0x7fb82406ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.819+0000 7fb81bfff640 1 -- 172.21.15.31:0/2855403432 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82406a650 con 0x7fb8240b3030 2024-01-19T04:56:44.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.820+0000 7fb81bfff640 1 --2- 172.21.15.31:0/2855403432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8240b3030 0x7fb824160f60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0097a0 tx=0x7fb80c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc7e49c45c74ca9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:44.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.820+0000 7fb8227fc640 1 -- 172.21.15.31:0/2855403432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c031830 con 0x7fb8240b3030 2024-01-19T04:56:44.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.820+0000 7fb8227fc640 1 -- 172.21.15.31:0/2855403432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c031990 con 0x7fb8240b3030 2024-01-19T04:56:44.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.820+0000 7fb8227fc640 1 -- 172.21.15.31:0/2855403432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c031d20 con 0x7fb8240b3030 2024-01-19T04:56:44.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 -- 172.21.15.31:0/2855403432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8240b3030 msgr2=0x7fb824160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:44.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2855403432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8240b3030 0x7fb824160f60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0097a0 tx=0x7fb80c02b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 -- 172.21.15.31:0/2855403432 shutdown_connections 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2855403432 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb824071df0 0x7fb8241614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2855403432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8240b3030 0x7fb824160f60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 -- 172.21.15.31:0/2855403432 >> 172.21.15.31:0/2855403432 conn(0x7fb82406d750 msgr2=0x7fb82406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 -- 172.21.15.31:0/2855403432 shutdown_connections 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.821+0000 7fb8293f1640 1 -- 172.21.15.31:0/2855403432 wait complete. 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.822+0000 7fb8293f1640 1 Processor -- start 2024-01-19T04:56:44.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.822+0000 7fb8293f1640 1 -- start start 2024-01-19T04:56:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824071df0 0x7fb82415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb8240b3030 0x7fb82414b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82415d300 0x7fb82414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82414c1f0 con 0x7fb82415d300 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82415bc00 con 0x7fb824071df0 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82415aff0 con 0x7fb8240b3030 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82415d300 0x7fb82414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82415d300 0x7fb82414bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59460/0 (socket says 172.21.15.31:59460) 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 -- 172.21.15.31:0/2619091679 learned_addr learned my addr 172.21.15.31:0/2619091679 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb822ffd640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824071df0 0x7fb82415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 -- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb8240b3030 msgr2=0x7fb82414b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb8240b3030 0x7fb82414b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 -- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824071df0 msgr2=0x7fb82415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824071df0 0x7fb82415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 -- 172.21.15.31:0/2619091679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c009400 con 0x7fb82415d300 2024-01-19T04:56:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb822ffd640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824071df0 0x7fb82415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8237fe640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82415d300 0x7fb82414bcb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb81c007e30 tx=0x7fb81c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb81b7fe640 1 -- 172.21.15.31:0/2619091679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c014620 con 0x7fb82415d300 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb81b7fe640 1 -- 172.21.15.31:0/2619091679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00ee90 con 0x7fb82415d300 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb81b7fe640 1 -- 172.21.15.31:0/2619091679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0149c0 con 0x7fb82415d300 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82415b2d0 con 0x7fb82415d300 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.823+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82415b890 con 0x7fb82415d300 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.824+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82415bd90 con 0x7fb82415d300 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.825+0000 7fb81b7fe640 1 -- 172.21.15.31:0/2619091679 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb81c014b20 con 0x7fb82415d300 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.825+0000 7fb81b7fe640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb80406aed0 0x7fb80406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:44.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.825+0000 7fb81b7fe640 1 -- 172.21.15.31:0/2619091679 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb81c08a230 con 0x7fb82415d300 2024-01-19T04:56:44.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.826+0000 7fb822ffd640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb80406aed0 0x7fb80406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:44.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.826+0000 7fb822ffd640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb80406aed0 0x7fb80406d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb814005880 tx=0x7fb814009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:44.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:44.830+0000 7fb81b7fe640 1 -- 172.21.15.31:0/2619091679 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb81c054ed0 con 0x7fb82415d300 2024-01-19T04:56:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.048+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb8240696e0 con 0x7fb82415d300 2024-01-19T04:56:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.048+0000 7fb81b7fe640 1 -- 172.21.15.31:0/2619091679 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+82 (secure 0 0 0) 0x7fb81c0584f0 con 0x7fb82415d300 2024-01-19T04:56:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb80406aed0 msgr2=0x7fb80406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb80406aed0 0x7fb80406d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb814005880 tx=0x7fb814009040 comp rx=0 tx=0).stop 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82415d300 msgr2=0x7fb82414bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82415d300 0x7fb82414bcb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb81c007e30 tx=0x7fb81c00e4b0 comp rx=0 tx=0).stop 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 shutdown_connections 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb80406aed0 0x7fb80406d390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82415d300 0x7fb82414bcb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:45.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb8240b3030 0x7fb82414b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 --2- 172.21.15.31:0/2619091679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824071df0 0x7fb82415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.050+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 >> 172.21.15.31:0/2619091679 conn(0x7fb82406d750 msgr2=0x7fb8240ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.051+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 shutdown_connections 2024-01-19T04:56:45.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45.051+0000 7fb8293f1640 1 -- 172.21.15.31:0/2619091679 wait complete. 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:56:45.061 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:45.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.061 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:56:45.076 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:56:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:56:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42688sec preferred_lft 42688sec 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c57:fcff:feba:867a/64 scope link 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::98a9:56ff:fef7:e409/64 scope link 2024-01-19T04:56:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:45.079 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:56:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::90b1:87ff:fefd:d7a/64 scope link 2024-01-19T04:56:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:45.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.079 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:56:45.142 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:56:45.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.144 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:56:45.199 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:56:45.199 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:56:45.199 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:56:45.199 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:56:45.200 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:56:45.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.200 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:56:45.200 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:56:45.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:56:45.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:56:45.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:56:45.200 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:56:45.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.294 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:56:45.309 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-19T04:56:45.309 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:56:45.309 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:45.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-01-19T04:56:45.376 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-19T04:56:45.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:56:45.376 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:45.431 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:45.431 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:56:45.432 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:45.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:56:45.553 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:56:45.609 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:56:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:45.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.611 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:56:45.664 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:56:45.664 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-19T04:56:45.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:56:45.733 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:56:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:56:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:45.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.791 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:56:45.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-01-19T04:56:45.792+0000 7f8182a38480 -1 init, newargv = 0x5589cdeb5b50 newargc=15 2024-01-19T04:56:45.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[52197]: starting ceph client 2024-01-19T04:56:45.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[52197]: starting fuse 2024-01-19T04:56:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:56:45.807 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:56:45.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:56:45.807 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:56:45.807 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:56:45.807 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:56:45.808 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:56:45.809 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:56:45.809 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:56:45.809 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:56:45.809 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:56:45.809 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:45.809 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-01-19T04:56:45.809 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:56:45.927 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.52197.asok 2024-01-19T04:56:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:56:45.937 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52197.asok 2024-01-19T04:56:45.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:45.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52197.asok status 2024-01-19T04:56:46.074 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:56:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:56:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:56:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:56:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "52197", 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5547, 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 5547 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:56:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2772380679 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2772380679 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.5547 192.168.0.1:0/2772380679", 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2772380679", 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 15, 2024-01-19T04:56:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 37, 2024-01-19T04:56:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:56:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:56:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:56:46.077 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:56:46.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:46.083 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:56:46.083 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:46.097 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:56:46.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:46.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:46.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:46.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:46.165 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-01-19T04:56:46.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:56:46.230 INFO:teuthology.orchestra.run.smithi031.stderr:setfattr: layoutfile: Permission denied 2024-01-19T04:56:46.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:46.230 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-01-19T04:56:46.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:56:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:setfattr: layoutdir: Permission denied 2024-01-19T04:56:46.286 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:46.286 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:46.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:46.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.482+0000 7fd8e91d0640 1 Processor -- start 2024-01-19T04:56:46.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8e91d0640 1 -- start start 2024-01-19T04:56:46.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8e91d0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e40cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8e91d0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8e40c9520 0x7fd8e40cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8e91d0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e40cbce0 con 0x7fd8e415d610 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8e91d0640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e40cd300 con 0x7fd8e40c9520 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8e91d0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4161400 con 0x7fd8e415d230 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8dbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd8e415d610 0x7fd8e406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58518/0 (socket says 172.21.15.31:58518) 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.483+0000 7fd8e359e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8e40c9520 0x7fd8e40cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8dbfff640 1 -- 172.21.15.31:0/1234819980 learned_addr learned my addr 172.21.15.31:0/1234819980 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e2d9d640 1 --2- 172.21.15.31:0/1234819980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e40cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e259c640 1 -- 172.21.15.31:0/1234819980 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143013489 0 0) 0x7fd8e40cbce0 con 0x7fd8e415d610 2024-01-19T04:56:46.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e259c640 1 -- 172.21.15.31:0/1234819980 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d0003520 con 0x7fd8e415d610 2024-01-19T04:56:46.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e359e640 1 -- 172.21.15.31:0/1234819980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 msgr2=0x7fd8e40cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:46.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e359e640 1 --2- 172.21.15.31:0/1234819980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e40cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e359e640 1 -- 172.21.15.31:0/1234819980 >> v1:172.21.15.31:6789/0 conn(0x7fd8e415d610 legacy=0x7fd8e406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e359e640 1 -- 172.21.15.31:0/1234819980 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e41616e0 con 0x7fd8e40c9520 2024-01-19T04:56:46.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.484+0000 7fd8e2d9d640 1 --2- 172.21.15.31:0/1234819980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e40cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e359e640 1 --2- 172.21.15.31:0/1234819980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e40cb7a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009d90 tx=0x7fd8dc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20feff1c0641b73d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e259c640 1 -- 172.21.15.31:0/1234819980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc02cc50 con 0x7fd8e40c9520 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e259c640 1 -- 172.21.15.31:0/1234819980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc02cdb0 con 0x7fd8e40c9520 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e259c640 1 -- 172.21.15.31:0/1234819980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc034660 con 0x7fd8e40c9520 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1234819980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 msgr2=0x7fd8e40cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1234819980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e40cb7a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009d90 tx=0x7fd8dc02bea0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1234819980 shutdown_connections 2024-01-19T04:56:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1234819980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e40cb7a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1234819980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e40cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1234819980 >> 172.21.15.31:0/1234819980 conn(0x7fd8e40cc1b0 msgr2=0x7fd8e40cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1234819980 shutdown_connections 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.485+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1234819980 wait complete. 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 Processor -- start 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 -- start start 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e4145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8e415d610 0x7fd8e41579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e40b3e40 con 0x7fd8e40c9520 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e40b2da0 con 0x7fd8e415d610 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e91d0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e40b3080 con 0x7fd8e415d230 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e4145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e4145080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59470/0 (socket says 172.21.15.31:59470) 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.487+0000 7fd8e2d9d640 1 -- 172.21.15.31:0/1184395722 learned_addr learned my addr 172.21.15.31:0/1184395722 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e359e640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8e415d610 0x7fd8e41579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8dbfff640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e2d9d640 1 -- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 msgr2=0x7fd8e406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e2d9d640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e406ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e2d9d640 1 -- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8e415d610 msgr2=0x7fd8e41579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e2d9d640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8e415d610 0x7fd8e41579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e2d9d640 1 -- 172.21.15.31:0/1184395722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc009a70 con 0x7fd8e40c9520 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e2d9d640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e4145080 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400e730 tx=0x7fd8d400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8db7fe640 1 -- 172.21.15.31:0/1184395722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4004230 con 0x7fd8e40c9520 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8db7fe640 1 -- 172.21.15.31:0/1184395722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4004390 con 0x7fd8e40c9520 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8db7fe640 1 -- 172.21.15.31:0/1184395722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d400c8f0 con 0x7fd8e40c9520 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e40b3360 con 0x7fd8e40c9520 2024-01-19T04:56:46.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.488+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e40b3920 con 0x7fd8e40c9520 2024-01-19T04:56:46.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.489+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e40696e0 con 0x7fd8e40c9520 2024-01-19T04:56:46.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.490+0000 7fd8db7fe640 1 -- 172.21.15.31:0/1184395722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd8d400ca50 con 0x7fd8e40c9520 2024-01-19T04:56:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.490+0000 7fd8db7fe640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8c406aed0 0x7fd8c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.490+0000 7fd8db7fe640 1 -- 172.21.15.31:0/1184395722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd8d4089380 con 0x7fd8e40c9520 2024-01-19T04:56:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.490+0000 7fd8dbfff640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8c406aed0 0x7fd8c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.491+0000 7fd8dbfff640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8c406aed0 0x7fd8c406d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40ae5e0 tx=0x7fd8cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.494+0000 7fd8db7fe640 1 -- 172.21.15.31:0/1184395722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd8d4054150 con 0x7fd8e40c9520 2024-01-19T04:56:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.674+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8e40cbf90 con 0x7fd8e40c9520 2024-01-19T04:56:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.675+0000 7fd8db7fe640 1 -- 172.21.15.31:0/1184395722 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7fd8d4057770 con 0x7fd8e40c9520 2024-01-19T04:56:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:46.676 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:46.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8c406aed0 msgr2=0x7fd8c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8c406aed0 0x7fd8c406d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40ae5e0 tx=0x7fd8cc009040 comp rx=0 tx=0).stop 2024-01-19T04:56:46.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 msgr2=0x7fd8e4145080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e4145080 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400e730 tx=0x7fd8d400ebf0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 shutdown_connections 2024-01-19T04:56:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd8c406aed0 0x7fd8c406d390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8e415d610 0x7fd8e41579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8e415d230 0x7fd8e406ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 --2- 172.21.15.31:0/1184395722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8e40c9520 0x7fd8e4145080 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 >> 172.21.15.31:0/1184395722 conn(0x7fd8e40cc1b0 msgr2=0x7fd8e4070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 shutdown_connections 2024-01-19T04:56:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.677+0000 7fd8e91d0640 1 -- 172.21.15.31:0/1184395722 wait complete. 2024-01-19T04:56:46.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:56:23.785927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:56:24.240600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-01-19T04:56:32.396495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:46.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.792+0000 7f58bec4f640 1 Processor -- start 2024-01-19T04:56:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.792+0000 7f58bec4f640 1 -- start start 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.793+0000 7f58bec4f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f58b815d720 0x7f58b80d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.793+0000 7f58bec4f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d1c20 0x7f58b80b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.793+0000 7f58bec4f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b80cf160 con 0x7f58b80d3ea0 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.793+0000 7f58bec4f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b80ce8c0 con 0x7f58b815d720 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.793+0000 7f58bec4f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80ceba0 con 0x7f58b80d1c20 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bc9eb640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f58b815d720 0x7f58b80d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bd1ec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d1c20 0x7f58b80b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58b7fff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f58b80d3ea0 0x7f58b80d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:41670/0 (socket says 172.21.15.31:41670) 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58b7fff640 1 -- 172.21.15.31:0/611283307 learned_addr learned my addr 172.21.15.31:0/611283307 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:46.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bc9eb640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f58b815d720 0x7f58b80d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:35022/0 (socket says 172.21.15.31:35022) 2024-01-19T04:56:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bc9eb640 1 -- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d1c20 msgr2=0x7f58b80b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bc9eb640 1 --2- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d1c20 0x7f58b80b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bc9eb640 1 -- 172.21.15.31:0/611283307 >> v1:172.21.15.137:6789/0 conn(0x7f58b80d3ea0 legacy=0x7f58b80d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58b77fe640 1 -- 172.21.15.31:0/611283307 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923372892 0 0) 0x7f58b80cf160 con 0x7f58b80d3ea0 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bc9eb640 1 -- 172.21.15.31:0/611283307 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80cee80 con 0x7f58b815d720 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.794+0000 7f58bd1ec640 1 --2- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d1c20 0x7f58b80b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.795+0000 7f58bc9eb640 1 --2- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 0x7f58b80d0ee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58b80cf350 tx=0x7f58a802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb728bf66cf2855 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.795+0000 7f58b77fe640 1 -- 172.21.15.31:0/611283307 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a80316f0 con 0x7f58b815d720 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.795+0000 7f58b77fe640 1 -- 172.21.15.31:0/611283307 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a8031850 con 0x7f58b815d720 2024-01-19T04:56:46.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.795+0000 7f58b77fe640 1 -- 172.21.15.31:0/611283307 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8031be0 con 0x7f58b815d720 2024-01-19T04:56:46.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 -- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 msgr2=0x7f58b80d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 --2- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 0x7f58b80d0ee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58b80cf350 tx=0x7f58a802b790 comp rx=0 tx=0).stop 2024-01-19T04:56:46.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 -- 172.21.15.31:0/611283307 shutdown_connections 2024-01-19T04:56:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 --2- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d1c20 0x7f58b80b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 --2- 172.21.15.31:0/611283307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 0x7f58b80d0ee0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 -- 172.21.15.31:0/611283307 >> 172.21.15.31:0/611283307 conn(0x7f58b80d3310 msgr2=0x7f58b80d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 -- 172.21.15.31:0/611283307 shutdown_connections 2024-01-19T04:56:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.796+0000 7f58bec4f640 1 -- 172.21.15.31:0/611283307 wait complete. 2024-01-19T04:56:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.797+0000 7f58bec4f640 1 Processor -- start 2024-01-19T04:56:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.797+0000 7f58bec4f640 1 -- start start 2024-01-19T04:56:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bec4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b80d1c20 0x7f58b81f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bec4f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d3ea0 0x7f58b80c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bec4f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 0x7f58b80c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bec4f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c6e40 con 0x7f58b80d1c20 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bec4f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c79c0 con 0x7f58b815d720 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bec4f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c7090 con 0x7f58b80d3ea0 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b80d1c20 0x7f58b81f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b80d1c20 0x7f58b81f3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59482/0 (socket says 172.21.15.31:59482) 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 -- 172.21.15.31:0/725275057 learned_addr learned my addr 172.21.15.31:0/725275057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bd1ec640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 0x7f58b80c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58b7fff640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d3ea0 0x7f58b80c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 -- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d3ea0 msgr2=0x7f58b80c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d3ea0 0x7f58b80c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 -- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 msgr2=0x7f58b80c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 0x7f58b80c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 -- 172.21.15.31:0/725275057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b0009a70 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bc9eb640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b80d1c20 0x7f58b81f3780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f58a8000c00 tx=0x7f58a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58b57fa640 1 -- 172.21.15.31:0/725275057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8034680 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58b57fa640 1 -- 172.21.15.31:0/725275057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a80347e0 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a8009400 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.798+0000 7f58b57fa640 1 -- 172.21.15.31:0/725275057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8034b90 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.799+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b80c7650 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.800+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b8074900 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.800+0000 7f58b57fa640 1 -- 172.21.15.31:0/725275057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f58a803b040 con 0x7f58b80d1c20 2024-01-19T04:56:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.800+0000 7f58b57fa640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f58a406aed0 0x7f58a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:46.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.800+0000 7f58b57fa640 1 -- 172.21.15.31:0/725275057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f58a80ab4c0 con 0x7f58b80d1c20 2024-01-19T04:56:46.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.800+0000 7f58b7fff640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f58a406aed0 0x7f58a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:46.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.801+0000 7f58b7fff640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f58a406aed0 0x7f58a406d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58b815bc20 tx=0x7f58ac007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:46.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.804+0000 7f58b57fa640 1 -- 172.21.15.31:0/725275057 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f58a80763d0 con 0x7f58b80d1c20 2024-01-19T04:56:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.981+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58b80cf7d0 con 0x7f58b80d1c20 2024-01-19T04:56:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.981+0000 7f58b57fa640 1 -- 172.21.15.31:0/725275057 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58a80799f0 con 0x7f58b80d1c20 2024-01-19T04:56:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f58a406aed0 msgr2=0x7f58a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f58a406aed0 0x7f58a406d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58b815bc20 tx=0x7f58ac007d50 comp rx=0 tx=0).stop 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b80d1c20 msgr2=0x7f58b81f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b80d1c20 0x7f58b81f3780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f58a8000c00 tx=0x7f58a80046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 shutdown_connections 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f58a406aed0 0x7f58a406d390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b815d720 0x7f58b80c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58b80d3ea0 0x7f58b80c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 --2- 172.21.15.31:0/725275057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b80d1c20 0x7f58b81f3780 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 >> 172.21.15.31:0/725275057 conn(0x7f58b80d3310 msgr2=0x7f58b80d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 shutdown_connections 2024-01-19T04:56:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:46.983+0000 7f58bec4f640 1 -- 172.21.15.31:0/725275057 wait complete. 2024-01-19T04:56:46.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:47.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.098+0000 7f2dad4c2640 1 Processor -- start 2024-01-19T04:56:47.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.098+0000 7f2dad4c2640 1 -- start start 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2dad4c2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2da80d3e50 0x7f2da815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2dad4c2640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da815dee0 con 0x7f2da80d18e0 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2dad4c2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da815e1c0 con 0x7f2da80d0ef0 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2dad4c2640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da815e4a0 con 0x7f2da80d3e50 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da6ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2da80d18e0 0x7f2da80d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58532/0 (socket says 172.21.15.31:58532) 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da6ffd640 1 -- 172.21.15.31:0/118267782 learned_addr learned my addr 172.21.15.31:0/118267782 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da67fc640 1 --2- 172.21.15.31:0/118267782 >> v2:172.21.15.31:3300/0 conn(0x7f2da80d3e50 0x7f2da815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da5ffb640 1 -- 172.21.15.31:0/118267782 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613150633 0 0) 0x7f2da815dee0 con 0x7f2da80d18e0 2024-01-19T04:56:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da5ffb640 1 -- 172.21.15.31:0/118267782 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d94003520 con 0x7f2da80d18e0 2024-01-19T04:56:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da67fc640 1 -- 172.21.15.31:0/118267782 >> v1:172.21.15.137:6789/0 conn(0x7f2da80d0ef0 legacy=0x7f2da80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da67fc640 1 -- 172.21.15.31:0/118267782 >> v1:172.21.15.31:6789/0 conn(0x7f2da80d18e0 legacy=0x7f2da80d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.099+0000 7f2da67fc640 1 -- 172.21.15.31:0/118267782 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da815e780 con 0x7f2da80d3e50 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.100+0000 7f2da67fc640 1 --2- 172.21.15.31:0/118267782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d3e50 0x7f2da815d860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097a0 tx=0x7f2d9002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbd4c088ca3b675 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.100+0000 7f2da5ffb640 1 -- 172.21.15.31:0/118267782 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90030610 con 0x7f2da80d3e50 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.100+0000 7f2da5ffb640 1 -- 172.21.15.31:0/118267782 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90030770 con 0x7f2da80d3e50 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.100+0000 7f2da5ffb640 1 -- 172.21.15.31:0/118267782 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90030b00 con 0x7f2da80d3e50 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.101+0000 7f2dad4c2640 1 -- 172.21.15.31:0/118267782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d3e50 msgr2=0x7f2da815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.101+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/118267782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d3e50 0x7f2da815d860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097a0 tx=0x7f2d9002b960 comp rx=0 tx=0).stop 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.101+0000 7f2dad4c2640 1 -- 172.21.15.31:0/118267782 shutdown_connections 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.101+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/118267782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d3e50 0x7f2da815d860 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.101+0000 7f2dad4c2640 1 -- 172.21.15.31:0/118267782 >> 172.21.15.31:0/118267782 conn(0x7f2da809c600 msgr2=0x7f2da80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.101+0000 7f2dad4c2640 1 -- 172.21.15.31:0/118267782 shutdown_connections 2024-01-19T04:56:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.101+0000 7f2dad4c2640 1 -- 172.21.15.31:0/118267782 wait complete. 2024-01-19T04:56:47.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.102+0000 7f2dad4c2640 1 Processor -- start 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.102+0000 7f2dad4c2640 1 -- start start 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2dad4c2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2da80d0ef0 0x7f2da80cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2dad4c2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d18e0 0x7f2da80b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2dad4c2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2da80d3e50 0x7f2da80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2dad4c2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80cd250 con 0x7f2da80d18e0 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2dad4c2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81f41b0 con 0x7f2da80d0ef0 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2dad4c2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81f33f0 con 0x7f2da80d3e50 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d18e0 0x7f2da80b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d18e0 0x7f2da80b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59498/0 (socket says 172.21.15.31:59498) 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 -- 172.21.15.31:0/3061173945 learned_addr learned my addr 172.21.15.31:0/3061173945 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da6ffd640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2da80d0ef0 0x7f2da80cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 -- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2da80d3e50 msgr2=0x7f2da80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2da80d3e50 0x7f2da80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 -- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2da80d0ef0 msgr2=0x7f2da80cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2da80d0ef0 0x7f2da80cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 -- 172.21.15.31:0/3061173945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90009400 con 0x7f2da80d18e0 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da6ffd640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2da80d0ef0 0x7f2da80cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.103+0000 7f2da67fc640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d18e0 0x7f2da80b4130 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2d900098c0 tx=0x7f2d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.104+0000 7f2d9f7fe640 1 -- 172.21.15.31:0/3061173945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9003e070 con 0x7f2da80d18e0 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.104+0000 7f2d9f7fe640 1 -- 172.21.15.31:0/3061173945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90030e10 con 0x7f2da80d18e0 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.104+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da81f36d0 con 0x7f2da80d18e0 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.104+0000 7f2d9f7fe640 1 -- 172.21.15.31:0/3061173945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90039c50 con 0x7f2da80d18e0 2024-01-19T04:56:47.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.104+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da81f3c90 con 0x7f2da80d18e0 2024-01-19T04:56:47.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.105+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da8074900 con 0x7f2da80d18e0 2024-01-19T04:56:47.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.105+0000 7f2d9f7fe640 1 -- 172.21.15.31:0/3061173945 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2d9004b400 con 0x7f2da80d18e0 2024-01-19T04:56:47.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.105+0000 7f2d9f7fe640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d8806af00 0x7f2d8806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.106+0000 7f2d9f7fe640 1 -- 172.21.15.31:0/3061173945 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2d900abab0 con 0x7f2da80d18e0 2024-01-19T04:56:47.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.106+0000 7f2da6ffd640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d8806af00 0x7f2d8806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.106+0000 7f2da6ffd640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d8806af00 0x7f2d8806d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2da0007fd0 tx=0x7f2da0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.110+0000 7f2d9f7fe640 1 -- 172.21.15.31:0/3061173945 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2d90076930 con 0x7f2da80d18e0 2024-01-19T04:56:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.289+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2da80cf700 con 0x7f2da80d18e0 2024-01-19T04:56:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.289+0000 7f2d9f7fe640 1 -- 172.21.15.31:0/3061173945 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d900340a0 con 0x7f2da80d18e0 2024-01-19T04:56:47.289 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d8806af00 msgr2=0x7f2d8806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d8806af00 0x7f2d8806d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2da0007fd0 tx=0x7f2da0009040 comp rx=0 tx=0).stop 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d18e0 msgr2=0x7f2da80b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d18e0 0x7f2da80b4130 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2d900098c0 tx=0x7f2d900046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 shutdown_connections 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2d8806af00 0x7f2d8806d3c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2da80d3e50 0x7f2da80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2da80d18e0 0x7f2da80b4130 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 --2- 172.21.15.31:0/3061173945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2da80d0ef0 0x7f2da80cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 >> 172.21.15.31:0/3061173945 conn(0x7f2da809c600 msgr2=0x7f2da80afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 shutdown_connections 2024-01-19T04:56:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.291+0000 7f2dad4c2640 1 -- 172.21.15.31:0/3061173945 wait complete. 2024-01-19T04:56:47.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:47.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.405+0000 7f0d183de640 1 Processor -- start 2024-01-19T04:56:47.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.405+0000 7f0d183de640 1 -- start start 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d183de640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d1015d3a0 0x7f0d1006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d183de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d183de640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100cf9f0 con 0x7f0d1015d780 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d183de640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d100cfcd0 con 0x7f0d1015d3a0 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d183de640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100cffb0 con 0x7f0d100cf570 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d1617a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d1015d3a0 0x7f0d1006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d1697b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0d1015d780 0x7f0d100cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58542/0 (socket says 172.21.15.31:58542) 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d1697b640 1 -- 172.21.15.31:0/2900725185 learned_addr learned my addr 172.21.15.31:0/2900725185 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d15979640 1 --2- 172.21.15.31:0/2900725185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d1617a640 1 -- 172.21.15.31:0/2900725185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 msgr2=0x7f0d10161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d1617a640 1 --2- 172.21.15.31:0/2900725185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d1617a640 1 -- 172.21.15.31:0/2900725185 >> v1:172.21.15.31:6789/0 conn(0x7f0d1015d780 legacy=0x7f0d100cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d15178640 1 -- 172.21.15.31:0/2900725185 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674940334 0 0) 0x7f0d100cf9f0 con 0x7f0d1015d780 2024-01-19T04:56:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.406+0000 7f0d1617a640 1 -- 172.21.15.31:0/2900725185 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100d0290 con 0x7f0d1015d3a0 2024-01-19T04:56:47.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.407+0000 7f0d1617a640 1 --2- 172.21.15.31:0/2900725185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1006c4e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009790 tx=0x7f0d0c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5978e8aebc0631f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.407+0000 7f0d15178640 1 -- 172.21.15.31:0/2900725185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c031800 con 0x7f0d1015d3a0 2024-01-19T04:56:47.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.407+0000 7f0d15178640 1 -- 172.21.15.31:0/2900725185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c031960 con 0x7f0d1015d3a0 2024-01-19T04:56:47.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.407+0000 7f0d15178640 1 -- 172.21.15.31:0/2900725185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c031cf0 con 0x7f0d1015d3a0 2024-01-19T04:56:47.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.407+0000 7f0d183de640 1 -- 172.21.15.31:0/2900725185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 msgr2=0x7f0d1006c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.408+0000 7f0d183de640 1 --2- 172.21.15.31:0/2900725185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1006c4e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009790 tx=0x7f0d0c02b8a0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.408+0000 7f0d183de640 1 -- 172.21.15.31:0/2900725185 shutdown_connections 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.408+0000 7f0d183de640 1 --2- 172.21.15.31:0/2900725185 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.408+0000 7f0d183de640 1 --2- 172.21.15.31:0/2900725185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1006c4e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.408+0000 7f0d183de640 1 -- 172.21.15.31:0/2900725185 >> 172.21.15.31:0/2900725185 conn(0x7f0d100d1100 msgr2=0x7f0d100d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.408+0000 7f0d183de640 1 -- 172.21.15.31:0/2900725185 shutdown_connections 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.408+0000 7f0d183de640 1 -- 172.21.15.31:0/2900725185 wait complete. 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.409+0000 7f0d183de640 1 Processor -- start 2024-01-19T04:56:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.409+0000 7f0d183de640 1 -- start start 2024-01-19T04:56:47.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1014ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d1015d780 0x7f0d101507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d10155610 con 0x7f0d1015d3a0 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d10154a00 con 0x7f0d1015d780 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d10154ce0 con 0x7f0d100cf570 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1014ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1014ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59512/0 (socket says 172.21.15.31:59512) 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 -- 172.21.15.31:0/4270810428 learned_addr learned my addr 172.21.15.31:0/4270810428 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d1617a640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d1697b640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d1015d780 0x7f0d101507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 -- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 msgr2=0x7f0d10154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 -- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d1015d780 msgr2=0x7f0d101507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d1015d780 0x7f0d101507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 -- 172.21.15.31:0/4270810428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c0093a0 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d1617a640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d1697b640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d1015d780 0x7f0d101507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d15979640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1014ff10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c7f0 tx=0x7f0d0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d06ffd640 1 -- 172.21.15.31:0/4270810428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00004220 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d06ffd640 1 -- 172.21.15.31:0/4270810428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00004380 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d06ffd640 1 -- 172.21.15.31:0/4270810428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00007860 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d10154fc0 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.410+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d10154660 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.411+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d10074900 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.412+0000 7f0d06ffd640 1 -- 172.21.15.31:0/4270810428 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0d00007bd0 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.412+0000 7f0d06ffd640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0cf406aed0 0x7f0cf406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.412+0000 7f0d06ffd640 1 -- 172.21.15.31:0/4270810428 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0d00089aa0 con 0x7f0d1015d3a0 2024-01-19T04:56:47.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.412+0000 7f0d1617a640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0cf406aed0 0x7f0cf406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.413+0000 7f0d1617a640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0cf406aed0 0x7f0cf406d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c036000 tx=0x7f0d0c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.416+0000 7f0d06ffd640 1 -- 172.21.15.31:0/4270810428 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0d000547e0 con 0x7f0d1015d3a0 2024-01-19T04:56:47.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.598+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d10067030 con 0x7f0d1015d3a0 2024-01-19T04:56:47.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.598+0000 7f0d06ffd640 1 -- 172.21.15.31:0/4270810428 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d0001e020 con 0x7f0d1015d3a0 2024-01-19T04:56:47.598 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0cf406aed0 msgr2=0x7f0cf406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0cf406aed0 0x7f0cf406d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c036000 tx=0x7f0d0c03c040 comp rx=0 tx=0).stop 2024-01-19T04:56:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 msgr2=0x7f0d1014ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1014ff10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c7f0 tx=0x7f0d0000ccb0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 shutdown_connections 2024-01-19T04:56:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0cf406aed0 0x7f0cf406d390 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d1015d780 0x7f0d101507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1015d3a0 0x7f0d1014ff10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 --2- 172.21.15.31:0/4270810428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0d100cf570 0x7f0d10154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 >> 172.21.15.31:0/4270810428 conn(0x7f0d100d1100 msgr2=0x7f0d1015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 shutdown_connections 2024-01-19T04:56:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.600+0000 7f0d183de640 1 -- 172.21.15.31:0/4270810428 wait complete. 2024-01-19T04:56:47.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-19T04:56:47.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.714+0000 7fa95f1ec640 1 Processor -- start 2024-01-19T04:56:47.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95f1ec640 1 -- start start 2024-01-19T04:56:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95f1ec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9580d3ea0 0x7fa9580d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95f1ec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9580cf160 con 0x7fa95815d720 2024-01-19T04:56:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95f1ec640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9580ce8c0 con 0x7fa9580d1c20 2024-01-19T04:56:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95f1ec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9580ceba0 con 0x7fa9580d3ea0 2024-01-19T04:56:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95cf88640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa95815d720 0x7fa9580d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58554/0 (socket says 172.21.15.31:58554) 2024-01-19T04:56:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95d789640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9580d3ea0 0x7fa9580d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.715+0000 7fa95cf88640 1 -- 172.21.15.31:0/3620209813 learned_addr learned my addr 172.21.15.31:0/3620209813 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.716+0000 7fa95d789640 1 -- 172.21.15.31:0/3620209813 >> v1:172.21.15.137:6789/0 conn(0x7fa9580d1c20 legacy=0x7fa9580b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.716+0000 7fa95d789640 1 -- 172.21.15.31:0/3620209813 >> v1:172.21.15.31:6789/0 conn(0x7fa95815d720 legacy=0x7fa9580d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.716+0000 7fa95d789640 1 -- 172.21.15.31:0/3620209813 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580cee80 con 0x7fa9580d3ea0 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.716+0000 7fa9577fe640 1 -- 172.21.15.31:0/3620209813 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690808537 0 0) 0x7fa9580cf160 con 0x7fa95815d720 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95d789640 1 --2- 172.21.15.31:0/3620209813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d3ea0 0x7fa9580d0ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa950009d90 tx=0x7fa95002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd62600755b06f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa9577fe640 1 -- 172.21.15.31:0/3620209813 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95002cb40 con 0x7fa9580d3ea0 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa9577fe640 1 -- 172.21.15.31:0/3620209813 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95002cca0 con 0x7fa9580d3ea0 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa9577fe640 1 -- 172.21.15.31:0/3620209813 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950034660 con 0x7fa9580d3ea0 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95f1ec640 1 -- 172.21.15.31:0/3620209813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d3ea0 msgr2=0x7fa9580d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/3620209813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d3ea0 0x7fa9580d0ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa950009d90 tx=0x7fa95002bd90 comp rx=0 tx=0).stop 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95f1ec640 1 -- 172.21.15.31:0/3620209813 shutdown_connections 2024-01-19T04:56:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/3620209813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d3ea0 0x7fa9580d0ee0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95f1ec640 1 -- 172.21.15.31:0/3620209813 >> 172.21.15.31:0/3620209813 conn(0x7fa9580d3310 msgr2=0x7fa9580d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:47.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95f1ec640 1 -- 172.21.15.31:0/3620209813 shutdown_connections 2024-01-19T04:56:47.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.717+0000 7fa95f1ec640 1 -- 172.21.15.31:0/3620209813 wait complete. 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.718+0000 7fa95f1ec640 1 Processor -- start 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.718+0000 7fa95f1ec640 1 -- start start 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95f1ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d1c20 0x7fa9580b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95f1ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580d3ea0 0x7fa9580b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95f1ec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa95815d720 0x7fa9581f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95f1ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580c7350 con 0x7fa9580d1c20 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95f1ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95815af40 con 0x7fa9580d3ea0 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95f1ec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580b8d00 con 0x7fa95815d720 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95cf88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d1c20 0x7fa9580b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95cf88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d1c20 0x7fa9580b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59528/0 (socket says 172.21.15.31:59528) 2024-01-19T04:56:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.719+0000 7fa95cf88640 1 -- 172.21.15.31:0/2406563410 learned_addr learned my addr 172.21.15.31:0/2406563410 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa957fff640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580d3ea0 0x7fa9580b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95d789640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa95815d720 0x7fa9581f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95cf88640 1 -- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa95815d720 msgr2=0x7fa9581f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95cf88640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa95815d720 0x7fa9581f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95cf88640 1 -- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580d3ea0 msgr2=0x7fa9580b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95cf88640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580d3ea0 0x7fa9580b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95cf88640 1 -- 172.21.15.31:0/2406563410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa950009a70 con 0x7fa9580d1c20 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95cf88640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d1c20 0x7fa9580b0b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c460 tx=0x7fa94c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa9557fa640 1 -- 172.21.15.31:0/2406563410 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0074e0 con 0x7fa9580d1c20 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa9557fa640 1 -- 172.21.15.31:0/2406563410 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c007660 con 0x7fa9580d1c20 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95815a980 con 0x7fa9580d1c20 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa9557fa640 1 -- 172.21.15.31:0/2406563410 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004a20 con 0x7fa9580d1c20 2024-01-19T04:56:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.720+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9580c6f70 con 0x7fa9580d1c20 2024-01-19T04:56:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.721+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9580c7490 con 0x7fa9580d1c20 2024-01-19T04:56:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.722+0000 7fa9557fa640 1 -- 172.21.15.31:0/2406563410 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa94c004b80 con 0x7fa9580d1c20 2024-01-19T04:56:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.722+0000 7fa9557fa640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa93c06aed0 0x7fa93c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.722+0000 7fa9557fa640 1 -- 172.21.15.31:0/2406563410 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fa94c088bc0 con 0x7fa9580d1c20 2024-01-19T04:56:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.722+0000 7fa957fff640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa93c06aed0 0x7fa93c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.723+0000 7fa957fff640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa93c06aed0 0x7fa93c06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa944004ea0 tx=0x7fa944009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:47.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.726+0000 7fa9557fa640 1 -- 172.21.15.31:0/2406563410 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa9580c7490 con 0x7fa9580d1c20 2024-01-19T04:56:47.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.904+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fa9580c7490 con 0x7fa9580d1c20 2024-01-19T04:56:47.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.905+0000 7fa9557fa640 1 -- 172.21.15.31:0/2406563410 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7fa9580c7490 con 0x7fa9580d1c20 2024-01-19T04:56:47.905 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa93c06aed0 msgr2=0x7fa93c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa93c06aed0 0x7fa93c06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa944004ea0 tx=0x7fa944009040 comp rx=0 tx=0).stop 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d1c20 msgr2=0x7fa9580b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d1c20 0x7fa9580b0b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c460 tx=0x7fa94c00c920 comp rx=0 tx=0).stop 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 shutdown_connections 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa93c06aed0 0x7fa93c06d390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa95815d720 0x7fa9581f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580d3ea0 0x7fa9580b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 --2- 172.21.15.31:0/2406563410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9580d1c20 0x7fa9580b0b50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 >> 172.21.15.31:0/2406563410 conn(0x7fa9580d3310 msgr2=0x7fa95815c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:47.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 shutdown_connections 2024-01-19T04:56:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:47.907+0000 7fa95f1ec640 1 -- 172.21.15.31:0/2406563410 wait complete. 2024-01-19T04:56:47.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:48.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.020+0000 7f960b8db640 1 Processor -- start 2024-01-19T04:56:48.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.020+0000 7f960b8db640 1 -- start start 2024-01-19T04:56:48.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f960b8db640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f96040ce090 0x7f96040d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f960b8db640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f96040cd2e0 0x7f96040b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f960b8db640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96040cce50 con 0x7f960415b6b0 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f960b8db640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040d03b0 con 0x7f96040ce090 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f960b8db640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040cc240 con 0x7f96040cd2e0 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9608e76640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f96040ce090 0x7f96040d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9609677640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f960415b6b0 0x7f96040cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58568/0 (socket says 172.21.15.31:58568) 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9609677640 1 -- 172.21.15.31:0/4092298037 learned_addr learned my addr 172.21.15.31:0/4092298037 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9609e78640 1 --2- 172.21.15.31:0/4092298037 >> v2:172.21.15.137:3300/0 conn(0x7f96040cd2e0 0x7f96040b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9608e76640 1 -- 172.21.15.31:0/4092298037 >> v2:172.21.15.137:3300/0 conn(0x7f96040cd2e0 msgr2=0x7f96040b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9608e76640 1 --2- 172.21.15.31:0/4092298037 >> v2:172.21.15.137:3300/0 conn(0x7f96040cd2e0 0x7f96040b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9608e76640 1 -- 172.21.15.31:0/4092298037 >> v1:172.21.15.31:6789/0 conn(0x7f960415b6b0 legacy=0x7f96040cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f95fbfff640 1 -- 172.21.15.31:0/4092298037 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136338388 0 0) 0x7f96040cce50 con 0x7f960415b6b0 2024-01-19T04:56:48.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.021+0000 7f9608e76640 1 -- 172.21.15.31:0/4092298037 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040cc520 con 0x7f96040ce090 2024-01-19T04:56:48.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.022+0000 7f9608e76640 1 --2- 172.21.15.31:0/4092298037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96040ce090 0x7f96040d1c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f96040d1090 tx=0x7f95f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d295c4cf832300 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.022+0000 7f95fbfff640 1 -- 172.21.15.31:0/4092298037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4031830 con 0x7f96040ce090 2024-01-19T04:56:48.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.022+0000 7f95fbfff640 1 -- 172.21.15.31:0/4092298037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4031990 con 0x7f96040ce090 2024-01-19T04:56:48.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.022+0000 7f95fbfff640 1 -- 172.21.15.31:0/4092298037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4031d20 con 0x7f96040ce090 2024-01-19T04:56:48.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 -- 172.21.15.31:0/4092298037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96040ce090 msgr2=0x7f96040d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 --2- 172.21.15.31:0/4092298037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96040ce090 0x7f96040d1c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f96040d1090 tx=0x7f95f402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 -- 172.21.15.31:0/4092298037 shutdown_connections 2024-01-19T04:56:48.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 --2- 172.21.15.31:0/4092298037 >> v2:172.21.15.137:3300/0 conn(0x7f96040cd2e0 0x7f96040b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 --2- 172.21.15.31:0/4092298037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96040ce090 0x7f96040d1c20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 -- 172.21.15.31:0/4092298037 >> 172.21.15.31:0/4092298037 conn(0x7f96040d1100 msgr2=0x7f96040d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 -- 172.21.15.31:0/4092298037 shutdown_connections 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.023+0000 7f960b8db640 1 -- 172.21.15.31:0/4092298037 wait complete. 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.024+0000 7f960b8db640 1 Processor -- start 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.024+0000 7f960b8db640 1 -- start start 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f960b8db640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96040cd2e0 0x7f96041578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f960b8db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96040ce090 0x7f96041f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f960b8db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960415b6b0 0x7f96040b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f960b8db640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040c94f0 con 0x7f960415b6b0 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f960b8db640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96041f1810 con 0x7f96040ce090 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f960b8db640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96041f1af0 con 0x7f96040cd2e0 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960415b6b0 0x7f96040b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960415b6b0 0x7f96040b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59544/0 (socket says 172.21.15.31:59544) 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 -- 172.21.15.31:0/1022738563 learned_addr learned my addr 172.21.15.31:0/1022738563 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9608e76640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96040ce090 0x7f96041f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609677640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96040cd2e0 0x7f96041578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 -- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96040cd2e0 msgr2=0x7f96041578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96040cd2e0 0x7f96041578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 -- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96040ce090 msgr2=0x7f96041f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96040ce090 0x7f96041f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 -- 172.21.15.31:0/1022738563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009400 con 0x7f960415b6b0 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609677640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96040cd2e0 0x7f96041578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f9609e78640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960415b6b0 0x7f96040b49f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007e40 tx=0x7f95fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f95f9ffb640 1 -- 172.21.15.31:0/1022738563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc014620 con 0x7f960415b6b0 2024-01-19T04:56:48.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f95f9ffb640 1 -- 172.21.15.31:0/1022738563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc00ee90 con 0x7f960415b6b0 2024-01-19T04:56:48.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96040c9700 con 0x7f960415b6b0 2024-01-19T04:56:48.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.025+0000 7f95f9ffb640 1 -- 172.21.15.31:0/1022738563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0149e0 con 0x7f960415b6b0 2024-01-19T04:56:48.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.026+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9604157e20 con 0x7f960415b6b0 2024-01-19T04:56:48.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.027+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96041f1cb0 con 0x7f960415b6b0 2024-01-19T04:56:48.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.027+0000 7f95f9ffb640 1 -- 172.21.15.31:0/1022738563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f95fc014b40 con 0x7f960415b6b0 2024-01-19T04:56:48.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.027+0000 7f95f9ffb640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95e806aed0 0x7f95e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.027+0000 7f95f9ffb640 1 -- 172.21.15.31:0/1022738563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f95fc08a1e0 con 0x7f960415b6b0 2024-01-19T04:56:48.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.028+0000 7f9609677640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95e806aed0 0x7f95e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.028+0000 7f9609677640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95e806aed0 0x7f95e806d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f96040d0bd0 tx=0x7f9600009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.032+0000 7f95f9ffb640 1 -- 172.21.15.31:0/1022738563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f95fc054f20 con 0x7f960415b6b0 2024-01-19T04:56:48.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.222+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9604074900 con 0x7f960415b6b0 2024-01-19T04:56:48.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.222+0000 7f95f9ffb640 1 -- 172.21.15.31:0/1022738563 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f95fc058540 con 0x7f960415b6b0 2024-01-19T04:56:48.222 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95e806aed0 msgr2=0x7f95e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95e806aed0 0x7f95e806d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f96040d0bd0 tx=0x7f9600009040 comp rx=0 tx=0).stop 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960415b6b0 msgr2=0x7f96040b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960415b6b0 0x7f96040b49f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007e40 tx=0x7f95fc00e4b0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 shutdown_connections 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f95e806aed0 0x7f95e806d390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960415b6b0 0x7f96040b49f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96040ce090 0x7f96041f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 --2- 172.21.15.31:0/1022738563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96040cd2e0 0x7f96041578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 >> 172.21.15.31:0/1022738563 conn(0x7f96040d1100 msgr2=0x7f960415d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:48.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 shutdown_connections 2024-01-19T04:56:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.224+0000 7f960b8db640 1 -- 172.21.15.31:0/1022738563 wait complete. 2024-01-19T04:56:48.234 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:48.234 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:48.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.391+0000 7fd5cde56640 1 Processor -- start 2024-01-19T04:56:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.391+0000 7fd5cde56640 1 -- start start 2024-01-19T04:56:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.391+0000 7fd5cde56640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5c80cfd10 0x7fd5c80d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.391+0000 7fd5cde56640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c80d0ef0 0x7fd5c80d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.391+0000 7fd5cde56640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c80d06b0 con 0x7fd5c8160bb0 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.391+0000 7fd5cde56640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c80d18c0 con 0x7fd5c80cfd10 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.391+0000 7fd5cde56640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80d1b90 con 0x7fd5c80d0ef0 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c6ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5c80cfd10 0x7fd5c80d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c77fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd5c8160bb0 0x7fd5c80b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58578/0 (socket says 172.21.15.31:58578) 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c77fe640 1 -- 172.21.15.31:0/166259201 learned_addr learned my addr 172.21.15.31:0/166259201 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c6ffd640 1 -- 172.21.15.31:0/166259201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c80d0ef0 msgr2=0x7fd5c80d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c7fff640 1 --2- 172.21.15.31:0/166259201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c80d0ef0 0x7fd5c80d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c6ffd640 1 --2- 172.21.15.31:0/166259201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c80d0ef0 0x7fd5c80d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c6ffd640 1 -- 172.21.15.31:0/166259201 >> v1:172.21.15.31:6789/0 conn(0x7fd5c8160bb0 legacy=0x7fd5c80b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c67fc640 1 -- 172.21.15.31:0/166259201 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416207887 0 0) 0x7fd5c80d06b0 con 0x7fd5c8160bb0 2024-01-19T04:56:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c6ffd640 1 -- 172.21.15.31:0/166259201 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80d1e70 con 0x7fd5c80cfd10 2024-01-19T04:56:48.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.392+0000 7fd5c7fff640 1 --2- 172.21.15.31:0/166259201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c80d0ef0 0x7fd5c80d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5c6ffd640 1 --2- 172.21.15.31:0/166259201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c80d0170 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0097a0 tx=0x7fd5bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41b4a5326ef4ea3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5c67fc640 1 -- 172.21.15.31:0/166259201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc031830 con 0x7fd5c80cfd10 2024-01-19T04:56:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5c67fc640 1 -- 172.21.15.31:0/166259201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc031990 con 0x7fd5c80cfd10 2024-01-19T04:56:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5c67fc640 1 -- 172.21.15.31:0/166259201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc031d20 con 0x7fd5c80cfd10 2024-01-19T04:56:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5cde56640 1 -- 172.21.15.31:0/166259201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 msgr2=0x7fd5c80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5cde56640 1 --2- 172.21.15.31:0/166259201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c80d0170 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0097a0 tx=0x7fd5bc02b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5cde56640 1 -- 172.21.15.31:0/166259201 shutdown_connections 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5cde56640 1 --2- 172.21.15.31:0/166259201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c80d0ef0 0x7fd5c80d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5cde56640 1 --2- 172.21.15.31:0/166259201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c80d0170 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.393+0000 7fd5cde56640 1 -- 172.21.15.31:0/166259201 >> 172.21.15.31:0/166259201 conn(0x7fd5c809c600 msgr2=0x7fd5c80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.394+0000 7fd5cde56640 1 -- 172.21.15.31:0/166259201 shutdown_connections 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.394+0000 7fd5cde56640 1 -- 172.21.15.31:0/166259201 wait complete. 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.395+0000 7fd5cde56640 1 Processor -- start 2024-01-19T04:56:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.395+0000 7fd5cde56640 1 -- start start 2024-01-19T04:56:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c81f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c80d0ef0 0x7fd5c806bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c8160bb0 0x7fd5c81f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c81f3c90 con 0x7fd5c80cfd10 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80ca3f0 con 0x7fd5c80d0ef0 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80ca690 con 0x7fd5c8160bb0 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c81f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c81f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59564/0 (socket says 172.21.15.31:59564) 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 -- 172.21.15.31:0/31183916 learned_addr learned my addr 172.21.15.31:0/31183916 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c6ffd640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c80d0ef0 0x7fd5c806bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c7fff640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c8160bb0 0x7fd5c81f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 -- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c8160bb0 msgr2=0x7fd5c81f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c8160bb0 0x7fd5c81f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 -- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c80d0ef0 msgr2=0x7fd5c806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c80d0ef0 0x7fd5c806bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 -- 172.21.15.31:0/31183916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc009400 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c6ffd640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c80d0ef0 0x7fd5c806bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c7fff640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c8160bb0 0x7fd5c81f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5c77fe640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c81f3750 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c430 tx=0x7fd5b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5affff640 1 -- 172.21.15.31:0/31183916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80074e0 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5affff640 1 -- 172.21.15.31:0/31183916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8007660 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c80ca970 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5affff640 1 -- 172.21.15.31:0/31183916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8004a20 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.396+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c80caf30 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.397+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c80696e0 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.398+0000 7fd5affff640 1 -- 172.21.15.31:0/31183916 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd5b8004b80 con 0x7fd5c80cfd10 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.398+0000 7fd5affff640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd5b406aed0 0x7fd5b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.398+0000 7fd5affff640 1 -- 172.21.15.31:0/31183916 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd5b8089660 con 0x7fd5c80cfd10 2024-01-19T04:56:48.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.398+0000 7fd5c6ffd640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd5b406aed0 0x7fd5b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.399+0000 7fd5c6ffd640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd5b406aed0 0x7fd5b406d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc036000 tx=0x7fd5bc03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.402+0000 7fd5affff640 1 -- 172.21.15.31:0/31183916 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd5b8054390 con 0x7fd5c80cfd10 2024-01-19T04:56:48.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.581+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5c808d3b0 con 0x7fd5c80cfd10 2024-01-19T04:56:48.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.582+0000 7fd5affff640 1 -- 172.21.15.31:0/31183916 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7fd5b80579b0 con 0x7fd5c80cfd10 2024-01-19T04:56:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:48.582 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd5b406aed0 msgr2=0x7fd5b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd5b406aed0 0x7fd5b406d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc036000 tx=0x7fd5bc03c040 comp rx=0 tx=0).stop 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 msgr2=0x7fd5c81f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c81f3750 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c430 tx=0x7fd5b800c8f0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 shutdown_connections 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd5b406aed0 0x7fd5b406d390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5c8160bb0 0x7fd5c81f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c80d0ef0 0x7fd5c806bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 --2- 172.21.15.31:0/31183916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5c80cfd10 0x7fd5c81f3750 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 >> 172.21.15.31:0/31183916 conn(0x7fd5c809c600 msgr2=0x7fd5c815d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 shutdown_connections 2024-01-19T04:56:48.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.584+0000 7fd5cde56640 1 -- 172.21.15.31:0/31183916 wait complete. 2024-01-19T04:56:48.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:56:23.785927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:56:24.240600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-01-19T04:56:32.396495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:48.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.699+0000 7f054aa31640 1 Processor -- start 2024-01-19T04:56:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.699+0000 7f054aa31640 1 -- start start 2024-01-19T04:56:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f054aa31640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d3a0 0x7f054406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f054aa31640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f05440cf570 0x7f0544161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f054aa31640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05440cf9f0 con 0x7f054415d780 2024-01-19T04:56:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f054aa31640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05440cfcd0 con 0x7f05440cf570 2024-01-19T04:56:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f054aa31640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440cffb0 con 0x7f054415d3a0 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f05437fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f05440cf570 0x7f0544161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f0543fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d3a0 0x7f054406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f0548fce640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f054415d780 0x7f05440cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:41714/0 (socket says 172.21.15.31:41714) 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.700+0000 7f0548fce640 1 -- 172.21.15.31:0/2590002464 learned_addr learned my addr 172.21.15.31:0/2590002464 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.701+0000 7f0543fff640 1 -- 172.21.15.31:0/2590002464 >> v2:172.21.15.137:3300/0 conn(0x7f05440cf570 msgr2=0x7f0544161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.701+0000 7f0543fff640 1 --2- 172.21.15.31:0/2590002464 >> v2:172.21.15.137:3300/0 conn(0x7f05440cf570 0x7f0544161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.701+0000 7f0543fff640 1 -- 172.21.15.31:0/2590002464 >> v1:172.21.15.137:6789/0 conn(0x7f054415d780 legacy=0x7f05440cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.701+0000 7f0543fff640 1 -- 172.21.15.31:0/2590002464 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440d0290 con 0x7f054415d3a0 2024-01-19T04:56:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.701+0000 7f05437fe640 1 --2- 172.21.15.31:0/2590002464 >> v2:172.21.15.137:3300/0 conn(0x7f05440cf570 0x7f0544161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:48.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f0543fff640 1 --2- 172.21.15.31:0/2590002464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d3a0 0x7f054406c4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0538009790 tx=0x7f053802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13dda7012a881f51 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f0542ffd640 1 -- 172.21.15.31:0/2590002464 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538031800 con 0x7f054415d3a0 2024-01-19T04:56:48.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f0542ffd640 1 -- 172.21.15.31:0/2590002464 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538031960 con 0x7f054415d3a0 2024-01-19T04:56:48.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f0542ffd640 1 -- 172.21.15.31:0/2590002464 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538031cf0 con 0x7f054415d3a0 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 -- 172.21.15.31:0/2590002464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d3a0 msgr2=0x7f054406c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 --2- 172.21.15.31:0/2590002464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d3a0 0x7f054406c4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0538009790 tx=0x7f053802b8a0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 -- 172.21.15.31:0/2590002464 shutdown_connections 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 --2- 172.21.15.31:0/2590002464 >> v2:172.21.15.137:3300/0 conn(0x7f05440cf570 0x7f0544161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 --2- 172.21.15.31:0/2590002464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d3a0 0x7f054406c4e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 -- 172.21.15.31:0/2590002464 >> 172.21.15.31:0/2590002464 conn(0x7f05440d1100 msgr2=0x7f05440d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 -- 172.21.15.31:0/2590002464 shutdown_connections 2024-01-19T04:56:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.702+0000 7f054aa31640 1 -- 172.21.15.31:0/2590002464 wait complete. 2024-01-19T04:56:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.703+0000 7f054aa31640 1 Processor -- start 2024-01-19T04:56:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f054aa31640 1 -- start start 2024-01-19T04:56:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f054aa31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05440cf570 0x7f05440c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f054aa31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054415d3a0 0x7f05440b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f054aa31640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d780 0x7f05440b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f054aa31640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440b5750 con 0x7f054415d3a0 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f054aa31640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440b5970 con 0x7f05440cf570 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f054aa31640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440b5c50 con 0x7f054415d780 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f05437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054415d3a0 0x7f05440b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f05437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054415d3a0 0x7f05440b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59574/0 (socket says 172.21.15.31:59574) 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.704+0000 7f05437fe640 1 -- 172.21.15.31:0/64067994 learned_addr learned my addr 172.21.15.31:0/64067994 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f0543fff640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05440cf570 0x7f05440c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f0548fce640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d780 0x7f05440b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f05437fe640 1 -- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d780 msgr2=0x7f05440b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f05437fe640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d780 0x7f05440b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f05437fe640 1 -- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05440cf570 msgr2=0x7f05440c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f05437fe640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05440cf570 0x7f05440c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f05437fe640 1 -- 172.21.15.31:0/64067994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534009400 con 0x7f054415d3a0 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f05437fe640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054415d3a0 0x7f05440b4cd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f053400ebe0 tx=0x7f053400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f0540ff9640 1 -- 172.21.15.31:0/64067994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534009760 con 0x7f054415d3a0 2024-01-19T04:56:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f0540ff9640 1 -- 172.21.15.31:0/64067994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053400cc30 con 0x7f054415d3a0 2024-01-19T04:56:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05380093a0 con 0x7f054415d3a0 2024-01-19T04:56:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f0540ff9640 1 -- 172.21.15.31:0/64067994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400eee0 con 0x7f054415d3a0 2024-01-19T04:56:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.705+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05440b6210 con 0x7f054415d3a0 2024-01-19T04:56:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.706+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05440ceb20 con 0x7f054415d3a0 2024-01-19T04:56:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.707+0000 7f0540ff9640 1 -- 172.21.15.31:0/64067994 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0534013470 con 0x7f054415d3a0 2024-01-19T04:56:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.707+0000 7f0540ff9640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f053006af00 0x7f053006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.707+0000 7f0540ff9640 1 -- 172.21.15.31:0/64067994 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0534089a40 con 0x7f054415d3a0 2024-01-19T04:56:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.707+0000 7f0543fff640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f053006af00 0x7f053006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.708+0000 7f0543fff640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f053006af00 0x7f053006d3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0538036000 tx=0x7f053803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:48.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.711+0000 7f0540ff9640 1 -- 172.21.15.31:0/64067994 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f053408be60 con 0x7f054415d3a0 2024-01-19T04:56:48.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.887+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0544074900 con 0x7f054415d3a0 2024-01-19T04:56:48.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.887+0000 7f0540ff9640 1 -- 172.21.15.31:0/64067994 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0534058210 con 0x7f054415d3a0 2024-01-19T04:56:48.887 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f053006af00 msgr2=0x7f053006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f053006af00 0x7f053006d3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0538036000 tx=0x7f053803c040 comp rx=0 tx=0).stop 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054415d3a0 msgr2=0x7f05440b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054415d3a0 0x7f05440b4cd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f053400ebe0 tx=0x7f053400c520 comp rx=0 tx=0).stop 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 shutdown_connections 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f053006af00 0x7f053006d3c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f054415d780 0x7f05440b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054415d3a0 0x7f05440b4cd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 --2- 172.21.15.31:0/64067994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05440cf570 0x7f05440c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 >> 172.21.15.31:0/64067994 conn(0x7f05440d1100 msgr2=0x7f0544159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 shutdown_connections 2024-01-19T04:56:48.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:48.889+0000 7f054aa31640 1 -- 172.21.15.31:0/64067994 wait complete. 2024-01-19T04:56:48.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:49.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.005+0000 7f846bb3e640 1 Processor -- start 2024-01-19T04:56:49.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.005+0000 7f846bb3e640 1 -- start start 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f846bb3e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f846415d340 0x7f84640cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f846bb3e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84640cc490 0x7f846406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f846bb3e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84640c6130 con 0x7f846406aaf0 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f846bb3e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84640cc910 con 0x7f84640cc490 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f846bb3e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846406af80 con 0x7f846415d340 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84690d9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84640cc490 0x7f846406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84690d9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84640cc490 0x7f846406a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59590/0 (socket says 172.21.15.31:59590) 2024-01-19T04:56:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84690d9640 1 -- 172.21.15.31:0/3669455119 learned_addr learned my addr 172.21.15.31:0/3669455119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84698da640 1 --2- 172.21.15.31:0/3669455119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f846415d340 0x7f84640cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84690d9640 1 -- 172.21.15.31:0/3669455119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f846415d340 msgr2=0x7f84640cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84690d9640 1 --2- 172.21.15.31:0/3669455119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f846415d340 0x7f84640cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84690d9640 1 -- 172.21.15.31:0/3669455119 >> v1:172.21.15.137:6789/0 conn(0x7f846406aaf0 legacy=0x7f8464160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.006+0000 7f84690d9640 1 -- 172.21.15.31:0/3669455119 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84640ce630 con 0x7f84640cc490 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.007+0000 7f84690d9640 1 --2- 172.21.15.31:0/3669455119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640cc490 0x7f846406a5b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84540098a0 tx=0x7f845402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc88a2dceada000e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.007+0000 7f84688d8640 1 -- 172.21.15.31:0/3669455119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84540317c0 con 0x7f84640cc490 2024-01-19T04:56:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.007+0000 7f84688d8640 1 -- 172.21.15.31:0/3669455119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8454031920 con 0x7f84640cc490 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.007+0000 7f84688d8640 1 -- 172.21.15.31:0/3669455119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454031cb0 con 0x7f84640cc490 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 -- 172.21.15.31:0/3669455119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640cc490 msgr2=0x7f846406a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3669455119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640cc490 0x7f846406a5b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84540098a0 tx=0x7f845402b860 comp rx=0 tx=0).stop 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 -- 172.21.15.31:0/3669455119 shutdown_connections 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3669455119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640cc490 0x7f846406a5b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3669455119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f846415d340 0x7f84640cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 -- 172.21.15.31:0/3669455119 >> 172.21.15.31:0/3669455119 conn(0x7f84640b1c50 msgr2=0x7f84640b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 -- 172.21.15.31:0/3669455119 shutdown_connections 2024-01-19T04:56:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.008+0000 7f846bb3e640 1 -- 172.21.15.31:0/3669455119 wait complete. 2024-01-19T04:56:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.009+0000 7f846bb3e640 1 Processor -- start 2024-01-19T04:56:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.009+0000 7f846bb3e640 1 -- start start 2024-01-19T04:56:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846bb3e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f846406aaf0 0x7f84641f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846bb3e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84640cc490 0x7f8464154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846bb3e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f846415d340 0x7f846415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846bb3e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641f2ef0 con 0x7f846415d340 2024-01-19T04:56:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846bb3e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641f3e80 con 0x7f846406aaf0 2024-01-19T04:56:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846bb3e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846415b2f0 con 0x7f84640cc490 2024-01-19T04:56:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f846415d340 0x7f846415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f846415d340 0x7f846415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59592/0 (socket says 172.21.15.31:59592) 2024-01-19T04:56:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 -- 172.21.15.31:0/3063393400 learned_addr learned my addr 172.21.15.31:0/3063393400 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f84690d9640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84640cc490 0x7f8464154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f84698da640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f846406aaf0 0x7f84641f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 -- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84640cc490 msgr2=0x7f8464154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84640cc490 0x7f8464154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 -- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f846406aaf0 msgr2=0x7f84641f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f846406aaf0 0x7f84641f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 -- 172.21.15.31:0/3063393400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540094d0 con 0x7f846415d340 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.010+0000 7f846a0db640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f846415d340 0x7f846415a930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f846000c710 tx=0x7f846000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.011+0000 7f845a7fc640 1 -- 172.21.15.31:0/3063393400 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460006d50 con 0x7f846415d340 2024-01-19T04:56:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.011+0000 7f845a7fc640 1 -- 172.21.15.31:0/3063393400 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846000ce20 con 0x7f846415d340 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.011+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846415b5d0 con 0x7f846415d340 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.011+0000 7f845a7fc640 1 -- 172.21.15.31:0/3063393400 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460003bb0 con 0x7f846415d340 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.011+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84641f3420 con 0x7f846415d340 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.012+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8464073d60 con 0x7f846415d340 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.012+0000 7f845a7fc640 1 -- 172.21.15.31:0/3063393400 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8460003d10 con 0x7f846415d340 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.012+0000 7f845a7fc640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f844806aed0 0x7f844806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.013+0000 7f845a7fc640 1 -- 172.21.15.31:0/3063393400 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f8460089300 con 0x7f846415d340 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.013+0000 7f84698da640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f844806aed0 0x7f844806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.013+0000 7f84698da640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f844806aed0 0x7f844806d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f845c006fd0 tx=0x7f845c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.017+0000 7f845a7fc640 1 -- 172.21.15.31:0/3063393400 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8460057b60 con 0x7f846415d340 2024-01-19T04:56:49.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.193+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f846415b7c0 con 0x7f846415d340 2024-01-19T04:56:49.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.194+0000 7f845a7fc640 1 -- 172.21.15.31:0/3063393400 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84600576f0 con 0x7f846415d340 2024-01-19T04:56:49.193 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f844806aed0 msgr2=0x7f844806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f844806aed0 0x7f844806d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f845c006fd0 tx=0x7f845c008040 comp rx=0 tx=0).stop 2024-01-19T04:56:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f846415d340 msgr2=0x7f846415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f846415d340 0x7f846415a930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f846000c710 tx=0x7f846000cbd0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 shutdown_connections 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f844806aed0 0x7f844806d390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f846415d340 0x7f846415a930 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84640cc490 0x7f8464154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 --2- 172.21.15.31:0/3063393400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f846406aaf0 0x7f84641f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 >> 172.21.15.31:0/3063393400 conn(0x7f84640b1c50 msgr2=0x7f8464073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 shutdown_connections 2024-01-19T04:56:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.196+0000 7f846bb3e640 1 -- 172.21.15.31:0/3063393400 wait complete. 2024-01-19T04:56:49.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:49.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.310+0000 7fd81079e640 1 Processor -- start 2024-01-19T04:56:49.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.310+0000 7fd81079e640 1 -- start start 2024-01-19T04:56:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.311+0000 7fd81079e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd80815d720 0x7fd8080d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.311+0000 7fd81079e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080cf160 con 0x7fd8080d3ea0 2024-01-19T04:56:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.311+0000 7fd81079e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080ce8c0 con 0x7fd8080d1c20 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.311+0000 7fd81079e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080ceba0 con 0x7fd80815d720 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.311+0000 7fd80dd39640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd8080d3ea0 0x7fd8080d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58592/0 (socket says 172.21.15.31:58592) 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.311+0000 7fd80dd39640 1 -- 172.21.15.31:0/1098185974 learned_addr learned my addr 172.21.15.31:0/1098185974 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80e53a640 1 --2- 172.21.15.31:0/1098185974 >> v2:172.21.15.137:3300/0 conn(0x7fd80815d720 0x7fd8080d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858841903 0 0) 0x7fd8080cf160 con 0x7fd8080d3ea0 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f0003520 con 0x7fd8080d3ea0 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3223245921 0 0) 0x7fd7f0003520 con 0x7fd8080d3ea0 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 >> v2:172.21.15.137:3300/0 conn(0x7fd80815d720 msgr2=0x7fd8080d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 --2- 172.21.15.31:0/1098185974 >> v2:172.21.15.137:3300/0 conn(0x7fd80815d720 0x7fd8080d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 >> v1:172.21.15.137:6789/0 conn(0x7fd8080d1c20 legacy=0x7fd8080b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080cee80 con 0x7fd8080d3ea0 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fd7fc001670 con 0x7fd8080d3ea0 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 >> v1:172.21.15.31:6789/0 conn(0x7fd8080d3ea0 legacy=0x7fd8080d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7f0003970 0x7fd7f0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7f00047c0 0x7fd7f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00052a0 0x7fd7f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080ceba0 con 0x7fd7f00047c0 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f000c170 con 0x7fd7f00052a0 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f000c330 con 0x7fd7f0003970 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80dd39640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7f00047c0 0x7fd7f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80ed3b640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00052a0 0x7fd7f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80e53a640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7f0003970 0x7fd7f0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80dd39640 1 -- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7f0003970 msgr2=0x7fd7f0003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80dd39640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7f0003970 0x7fd7f0003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80dd39640 1 -- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00052a0 msgr2=0x7fd7f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80dd39640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00052a0 0x7fd7f000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80dd39640 1 -- 172.21.15.31:0/1098185974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f0003520 con 0x7fd7f00047c0 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80ed3b640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00052a0 0x7fd7f000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.312+0000 7fd80ed3b640 1 -- 172.21.15.31:0/1098185974 reap_dead start 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.313+0000 7fd80dd39640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7f00047c0 0x7fd7f0004c40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c210 tx=0x7fd7f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c9b57fba305492 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.313+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc005ad0 con 0x7fd7f00047c0 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.313+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc005c30 con 0x7fd7f00047c0 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.313+0000 7fd80d538640 1 -- 172.21.15.31:0/1098185974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc02cad0 con 0x7fd7f00047c0 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.314+0000 7fd81079e640 1 -- 172.21.15.31:0/1098185974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7f00047c0 msgr2=0x7fd7f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.314+0000 7fd81079e640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7f00047c0 0x7fd7f0004c40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c210 tx=0x7fd7f00057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.314+0000 7fd81079e640 1 -- 172.21.15.31:0/1098185974 shutdown_connections 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.314+0000 7fd81079e640 1 --2- 172.21.15.31:0/1098185974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7f00047c0 0x7fd7f0004c40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.314+0000 7fd81079e640 1 -- 172.21.15.31:0/1098185974 >> 172.21.15.31:0/1098185974 conn(0x7fd8080d3310 msgr2=0x7fd8080d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.314+0000 7fd81079e640 1 -- 172.21.15.31:0/1098185974 shutdown_connections 2024-01-19T04:56:49.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.314+0000 7fd81079e640 1 -- 172.21.15.31:0/1098185974 wait complete. 2024-01-19T04:56:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.315+0000 7fd81079e640 1 Processor -- start 2024-01-19T04:56:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.315+0000 7fd81079e640 1 -- start start 2024-01-19T04:56:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd81079e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00047c0 0x7fd808151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd81079e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8080c8020 0x7fd8080c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd81079e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8080b0e00 0x7fd8080b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd81079e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b17c0 con 0x7fd8080b0e00 2024-01-19T04:56:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd81079e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd808151cd0 con 0x7fd7f00047c0 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd81079e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c9b80 con 0x7fd8080c8020 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8080b0e00 0x7fd8080b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8080b0e00 0x7fd8080b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59604/0 (socket says 172.21.15.31:59604) 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 -- 172.21.15.31:0/3150869239 learned_addr learned my addr 172.21.15.31:0/3150869239 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80e53a640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00047c0 0x7fd808151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80dd39640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8080c8020 0x7fd8080c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 -- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8080c8020 msgr2=0x7fd8080c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8080c8020 0x7fd8080c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 -- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00047c0 msgr2=0x7fd808151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00047c0 0x7fd808151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 -- 172.21.15.31:0/3150869239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080c8ae0 con 0x7fd8080b0e00 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80dd39640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8080c8020 0x7fd8080c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.316+0000 7fd80ed3b640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8080b0e00 0x7fd8080b1280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd8081611c0 tx=0x7fd7f800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.317+0000 7fd806ffd640 1 -- 172.21.15.31:0/3150869239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f800eed0 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.317+0000 7fd806ffd640 1 -- 172.21.15.31:0/3150869239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f8003970 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.317+0000 7fd806ffd640 1 -- 172.21.15.31:0/3150869239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8003d20 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.317+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8080c8dc0 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.317+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8080c9380 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.318+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd808074900 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.318+0000 7fd806ffd640 1 -- 172.21.15.31:0/3150869239 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd7f80050d0 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.318+0000 7fd806ffd640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7f406aed0 0x7fd7f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.319+0000 7fd806ffd640 1 -- 172.21.15.31:0/3150869239 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd7f8089440 con 0x7fd8080b0e00 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.319+0000 7fd80e53a640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7f406aed0 0x7fd7f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.319+0000 7fd80e53a640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7f406aed0 0x7fd7f406d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd8080c9e40 tx=0x7fd800008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.323+0000 7fd806ffd640 1 -- 172.21.15.31:0/3150869239 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd7f8054180 con 0x7fd8080b0e00 2024-01-19T04:56:49.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.505+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8080d0ba0 con 0x7fd8080b0e00 2024-01-19T04:56:49.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.505+0000 7fd806ffd640 1 -- 172.21.15.31:0/3150869239 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7f80577a0 con 0x7fd8080b0e00 2024-01-19T04:56:49.505 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7f406aed0 msgr2=0x7fd7f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7f406aed0 0x7fd7f406d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd8080c9e40 tx=0x7fd800008040 comp rx=0 tx=0).stop 2024-01-19T04:56:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8080b0e00 msgr2=0x7fd8080b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8080b0e00 0x7fd8080b1280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd8081611c0 tx=0x7fd7f800eb90 comp rx=0 tx=0).stop 2024-01-19T04:56:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 shutdown_connections 2024-01-19T04:56:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd7f406aed0 0x7fd7f406d390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8080b0e00 0x7fd8080b1280 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd8080c8020 0x7fd8080c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 --2- 172.21.15.31:0/3150869239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7f00047c0 0x7fd808151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 >> 172.21.15.31:0/3150869239 conn(0x7fd8080d3310 msgr2=0x7fd8080b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 shutdown_connections 2024-01-19T04:56:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.507+0000 7fd81079e640 1 -- 172.21.15.31:0/3150869239 wait complete. 2024-01-19T04:56:49.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-19T04:56:49.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.622+0000 7f12777ec640 1 Processor -- start 2024-01-19T04:56:49.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f12777ec640 1 -- start start 2024-01-19T04:56:49.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f12777ec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f12700d3ea0 0x7f12700d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f12777ec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700cf160 con 0x7f12700d1c20 2024-01-19T04:56:49.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f12777ec640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700ce8c0 con 0x7f127015d720 2024-01-19T04:56:49.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f12777ec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12700ceba0 con 0x7f12700d3ea0 2024-01-19T04:56:49.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f1275d89640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f12700d3ea0 0x7f12700d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f1274d87640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f12700d1c20 0x7f12700b3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33898/0 (socket says 172.21.15.31:33898) 2024-01-19T04:56:49.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.623+0000 7f1274d87640 1 -- 172.21.15.31:0/2404101341 learned_addr learned my addr 172.21.15.31:0/2404101341 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:49.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.624+0000 7f1275d89640 1 -- 172.21.15.31:0/2404101341 >> v1:172.21.15.137:6789/0 conn(0x7f127015d720 legacy=0x7f12700d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:49.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.624+0000 7f1275d89640 1 -- 172.21.15.31:0/2404101341 >> v1:172.21.15.31:6789/0 conn(0x7f12700d1c20 legacy=0x7f12700b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.624+0000 7f1275d89640 1 -- 172.21.15.31:0/2404101341 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700cee80 con 0x7f12700d3ea0 2024-01-19T04:56:49.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.624+0000 7f126ffff640 1 -- 172.21.15.31:0/2404101341 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633690379 0 0) 0x7f12700cf160 con 0x7f12700d1c20 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.624+0000 7f1275d89640 1 --2- 172.21.15.31:0/2404101341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12700d3ea0 0x7f12700d0ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1260009d90 tx=0x7f126002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc90ad35caabf36 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f126ffff640 1 -- 172.21.15.31:0/2404101341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126002cbd0 con 0x7f12700d3ea0 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f126ffff640 1 -- 172.21.15.31:0/2404101341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126002cd30 con 0x7f12700d3ea0 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f126ffff640 1 -- 172.21.15.31:0/2404101341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126002cea0 con 0x7f12700d3ea0 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f12777ec640 1 -- 172.21.15.31:0/2404101341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12700d3ea0 msgr2=0x7f12700d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f12777ec640 1 --2- 172.21.15.31:0/2404101341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12700d3ea0 0x7f12700d0ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1260009d90 tx=0x7f126002be20 comp rx=0 tx=0).stop 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f12777ec640 1 -- 172.21.15.31:0/2404101341 shutdown_connections 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f12777ec640 1 --2- 172.21.15.31:0/2404101341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12700d3ea0 0x7f12700d0ee0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f12777ec640 1 -- 172.21.15.31:0/2404101341 >> 172.21.15.31:0/2404101341 conn(0x7f12700d3310 msgr2=0x7f12700d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f12777ec640 1 -- 172.21.15.31:0/2404101341 shutdown_connections 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.625+0000 7f12777ec640 1 -- 172.21.15.31:0/2404101341 wait complete. 2024-01-19T04:56:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.626+0000 7f12777ec640 1 Processor -- start 2024-01-19T04:56:49.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.626+0000 7f12777ec640 1 -- start start 2024-01-19T04:56:49.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f12777ec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12700d1c20 0x7f12701f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f12777ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12700d3ea0 0x7f12701f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f12777ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f127015d720 0x7f12701f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f12777ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701f4db0 con 0x7f127015d720 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f12777ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701f5090 con 0x7f12700d3ea0 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f12777ec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12701f5370 con 0x7f12700d1c20 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f127015d720 0x7f12701f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f127015d720 0x7f12701f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39702/0 (socket says 172.21.15.31:39702) 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 -- 172.21.15.31:0/2290461370 learned_addr learned my addr 172.21.15.31:0/2290461370 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1274d87640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12700d3ea0 0x7f12701f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 -- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12700d1c20 msgr2=0x7f12701f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12700d1c20 0x7f12701f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 -- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12700d3ea0 msgr2=0x7f12701f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12700d3ea0 0x7f12701f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.627+0000 7f1275d89640 1 -- 172.21.15.31:0/2290461370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1260009a70 con 0x7f127015d720 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.628+0000 7f1275d89640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f127015d720 0x7f12701f5e50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1260009d90 tx=0x7f1260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.628+0000 7f126dffb640 1 -- 172.21.15.31:0/2290461370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126002cea0 con 0x7f127015d720 2024-01-19T04:56:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.628+0000 7f126dffb640 1 -- 172.21.15.31:0/2290461370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1260040650 con 0x7f127015d720 2024-01-19T04:56:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.628+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12701f5650 con 0x7f127015d720 2024-01-19T04:56:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.628+0000 7f126dffb640 1 -- 172.21.15.31:0/2290461370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260040a00 con 0x7f127015d720 2024-01-19T04:56:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.628+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12701f4aa0 con 0x7f127015d720 2024-01-19T04:56:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.629+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12701f5b70 con 0x7f127015d720 2024-01-19T04:56:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.630+0000 7f126dffb640 1 -- 172.21.15.31:0/2290461370 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1260040b60 con 0x7f127015d720 2024-01-19T04:56:49.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.630+0000 7f126dffb640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f125c06aed0 0x7f125c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.630+0000 7f126dffb640 1 -- 172.21.15.31:0/2290461370 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f12600ac5f0 con 0x7f127015d720 2024-01-19T04:56:49.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.630+0000 7f1275588640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f125c06aed0 0x7f125c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.631+0000 7f1275588640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f125c06aed0 0x7f125c06d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1268000f60 tx=0x7f12680079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.634+0000 7f126dffb640 1 -- 172.21.15.31:0/2290461370 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f12701f5b70 con 0x7f127015d720 2024-01-19T04:56:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.812+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f12701f5b70 con 0x7f127015d720 2024-01-19T04:56:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.812+0000 7f126dffb640 1 -- 172.21.15.31:0/2290461370 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7f12701f5b70 con 0x7f127015d720 2024-01-19T04:56:49.812 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:49.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f125c06aed0 msgr2=0x7f125c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f125c06aed0 0x7f125c06d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1268000f60 tx=0x7f12680079b0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f127015d720 msgr2=0x7f12701f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f127015d720 0x7f12701f5e50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1260009d90 tx=0x7f1260004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:49.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 shutdown_connections 2024-01-19T04:56:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f125c06aed0 0x7f125c06d390 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f127015d720 0x7f12701f5e50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12700d3ea0 0x7f12701f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 --2- 172.21.15.31:0/2290461370 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12700d1c20 0x7f12701f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 >> 172.21.15.31:0/2290461370 conn(0x7f12700d3310 msgr2=0x7f12700d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 shutdown_connections 2024-01-19T04:56:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.814+0000 7f12777ec640 1 -- 172.21.15.31:0/2290461370 wait complete. 2024-01-19T04:56:49.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.927+0000 7f7501e75640 1 Processor -- start 2024-01-19T04:56:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.927+0000 7f7501e75640 1 -- start start 2024-01-19T04:56:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f7501e75640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc0cf570 0x7f74fc161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f7501e75640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc0cf9f0 con 0x7f74fc15d780 2024-01-19T04:56:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f7501e75640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc0cfcd0 con 0x7f74fc15d3a0 2024-01-19T04:56:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f7501e75640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0cffb0 con 0x7f74fc0cf570 2024-01-19T04:56:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74fbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f74fc15d780 0x7f74fc167480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33908/0 (socket says 172.21.15.31:33908) 2024-01-19T04:56:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74fbfff640 1 -- 172.21.15.31:0/2021888846 learned_addr learned my addr 172.21.15.31:0/2021888846 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74faffd640 1 --2- 172.21.15.31:0/2021888846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc0cf570 0x7f74fc161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122424997 0 0) 0x7f74fc0cf9f0 con 0x7f74fc15d780 2024-01-19T04:56:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e4003520 con 0x7f74fc15d780 2024-01-19T04:56:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094048544 0 0) 0x7f74fc0cfcd0 con 0x7f74fc15d3a0 2024-01-19T04:56:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc0cf9f0 con 0x7f74fc15d3a0 2024-01-19T04:56:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74faffd640 1 -- 172.21.15.31:0/2021888846 >> v1:172.21.15.137:6789/0 conn(0x7f74fc15d3a0 legacy=0x7f74fc06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74faffd640 1 -- 172.21.15.31:0/2021888846 >> v1:172.21.15.31:6789/0 conn(0x7f74fc15d780 legacy=0x7f74fc167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358843542 0 0) 0x7f74e4003520 con 0x7f74fc15d780 2024-01-19T04:56:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.928+0000 7f74faffd640 1 -- 172.21.15.31:0/2021888846 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc0d0290 con 0x7f74fc0cf570 2024-01-19T04:56:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.929+0000 7f74faffd640 1 --2- 172.21.15.31:0/2021888846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc0cf570 0x7f74fc161150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0097d0 tx=0x7f74ec02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4e2b760ed9542a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.929+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f40019f0 con 0x7f74fc0cf570 2024-01-19T04:56:49.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.929+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec02e050 con 0x7f74fc0cf570 2024-01-19T04:56:49.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.929+0000 7f74fa7fc640 1 -- 172.21.15.31:0/2021888846 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec030710 con 0x7f74fc0cf570 2024-01-19T04:56:49.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.930+0000 7f7501e75640 1 -- 172.21.15.31:0/2021888846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc0cf570 msgr2=0x7f74fc161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.930+0000 7f7501e75640 1 --2- 172.21.15.31:0/2021888846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc0cf570 0x7f74fc161150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0097d0 tx=0x7f74ec02b800 comp rx=0 tx=0).stop 2024-01-19T04:56:49.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.930+0000 7f7501e75640 1 -- 172.21.15.31:0/2021888846 shutdown_connections 2024-01-19T04:56:49.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.930+0000 7f7501e75640 1 --2- 172.21.15.31:0/2021888846 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc0cf570 0x7f74fc161150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.930+0000 7f7501e75640 1 -- 172.21.15.31:0/2021888846 >> 172.21.15.31:0/2021888846 conn(0x7f74fc0d1100 msgr2=0x7f74fc0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:49.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.930+0000 7f7501e75640 1 -- 172.21.15.31:0/2021888846 shutdown_connections 2024-01-19T04:56:49.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.930+0000 7f7501e75640 1 -- 172.21.15.31:0/2021888846 wait complete. 2024-01-19T04:56:49.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.931+0000 7f7501e75640 1 Processor -- start 2024-01-19T04:56:49.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.931+0000 7f7501e75640 1 -- start start 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f7501e75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74fc0cf570 0x7f74fc15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f7501e75640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc15d3a0 0x7f74fc0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f7501e75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74fc15d780 0x7f74fc0c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f7501e75640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc09e9e0 con 0x7f74fc0cf570 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f7501e75640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc09e550 con 0x7f74fc15d780 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f7501e75640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc09dcb0 con 0x7f74fc15d3a0 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74fc0cf570 0x7f74fc15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74fc0cf570 0x7f74fc15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39714/0 (socket says 172.21.15.31:39714) 2024-01-19T04:56:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 -- 172.21.15.31:0/2890296976 learned_addr learned my addr 172.21.15.31:0/2890296976 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fbfff640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74fc15d780 0x7f74fc0c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74faffd640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc15d3a0 0x7f74fc0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 -- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc15d3a0 msgr2=0x7f74fc0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc15d3a0 0x7f74fc0c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 -- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74fc15d780 msgr2=0x7f74fc0c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74fc15d780 0x7f74fc0c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74fb7fe640 1 -- 172.21.15.31:0/2890296976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec009400 con 0x7f74fc0cf570 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.932+0000 7f74faffd640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc15d3a0 0x7f74fc0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.933+0000 7f74fb7fe640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74fc0cf570 0x7f74fc15b3f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c500 tx=0x7f74f000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.933+0000 7f74e3fff640 1 -- 172.21.15.31:0/2890296976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f00052d0 con 0x7f74fc0cf570 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.933+0000 7f74e3fff640 1 -- 172.21.15.31:0/2890296976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f0005430 con 0x7f74fc0cf570 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.933+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc09df90 con 0x7f74fc0cf570 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.933+0000 7f74e3fff640 1 -- 172.21.15.31:0/2890296976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0003880 con 0x7f74fc0cf570 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.933+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74fc09af50 con 0x7f74fc0cf570 2024-01-19T04:56:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.934+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74fc074900 con 0x7f74fc0cf570 2024-01-19T04:56:49.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.934+0000 7f74e3fff640 1 -- 172.21.15.31:0/2890296976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f74f00039e0 con 0x7f74fc0cf570 2024-01-19T04:56:49.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.935+0000 7f74e3fff640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74e806aed0 0x7f74e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:49.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.935+0000 7f74e3fff640 1 -- 172.21.15.31:0/2890296976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f74f0089320 con 0x7f74fc0cf570 2024-01-19T04:56:49.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.935+0000 7f74faffd640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74e806aed0 0x7f74e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:49.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.935+0000 7f74faffd640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74e806aed0 0x7f74e806d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f74ec034000 tx=0x7f74ec008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:49.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:49.940+0000 7f74e3fff640 1 -- 172.21.15.31:0/2890296976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f74f0054130 con 0x7f74fc0cf570 2024-01-19T04:56:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.132+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74fc067030 con 0x7f74fc0cf570 2024-01-19T04:56:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.132+0000 7f74e3fff640 1 -- 172.21.15.31:0/2890296976 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f74f0057750 con 0x7f74fc0cf570 2024-01-19T04:56:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74e806aed0 msgr2=0x7f74e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74e806aed0 0x7f74e806d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f74ec034000 tx=0x7f74ec008030 comp rx=0 tx=0).stop 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74fc0cf570 msgr2=0x7f74fc15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74fc0cf570 0x7f74fc15b3f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c500 tx=0x7f74f000c9c0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 shutdown_connections 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f74e806aed0 0x7f74e806d390 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74fc15d780 0x7f74fc0c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74fc15d3a0 0x7f74fc0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 --2- 172.21.15.31:0/2890296976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74fc0cf570 0x7f74fc15b3f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 >> 172.21.15.31:0/2890296976 conn(0x7f74fc0d1100 msgr2=0x7f74fc06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 shutdown_connections 2024-01-19T04:56:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.134+0000 7f7501e75640 1 -- 172.21.15.31:0/2890296976 wait complete. 2024-01-19T04:56:50.144 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:56:50.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:50.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:50.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[52197]: fuse finished with error 0 and tester_r 0 2024-01-19T04:56:50.171 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:50.184 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:56:50.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:50.185 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:56:50.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-01-19T04:56:50.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.389+0000 7f6cfcf9e640 1 Processor -- start 2024-01-19T04:56:50.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.389+0000 7f6cfcf9e640 1 -- start start 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cfcf9e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6cf815b6b0 0x7f6cf80d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cfcf9e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80cce50 con 0x7f6cf80ce090 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cfcf9e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80cc240 con 0x7f6cf80cd2e0 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cfcf9e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80d03b0 con 0x7f6cf815b6b0 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf6d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6cf80ce090 0x7f6cf80b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33912/0 (socket says 172.21.15.31:33912) 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf6d9d640 1 -- 172.21.15.31:0/422772001 learned_addr learned my addr 172.21.15.31:0/422772001 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf659c640 1 --2- 172.21.15.31:0/422772001 >> v2:172.21.15.137:3300/0 conn(0x7f6cf815b6b0 0x7f6cf80d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205393688 0 0) 0x7f6cf80cce50 con 0x7f6cf80ce090 2024-01-19T04:56:50.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce4003520 con 0x7f6cf80ce090 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257178012 0 0) 0x7f6ce4003520 con 0x7f6cf80ce090 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 >> v2:172.21.15.137:3300/0 conn(0x7f6cf815b6b0 msgr2=0x7f6cf80d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 --2- 172.21.15.31:0/422772001 >> v2:172.21.15.137:3300/0 conn(0x7f6cf815b6b0 0x7f6cf80d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 >> v1:172.21.15.137:6789/0 conn(0x7f6cf80cd2e0 legacy=0x7f6cf80cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80cc520 con 0x7f6cf80ce090 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f6cf00019f0 con 0x7f6cf80ce090 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.390+0000 7f6cf659c640 1 --2- 172.21.15.31:0/422772001 >> v2:172.21.15.137:3300/0 conn(0x7f6cf815b6b0 0x7f6cf80d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 >> v1:172.21.15.31:6789/0 conn(0x7f6cf80ce090 legacy=0x7f6cf80b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf5d9b640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6ce4003970 0x7f6ce4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf5d9b640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf5d9b640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ce4005180 0x7f6ce400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80d03b0 con 0x7f6ce4005180 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce400c030 con 0x7f6ce40047c0 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce400c1f0 con 0x7f6ce4003970 2024-01-19T04:56:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf6d9d640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ce4005180 0x7f6ce400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6ceffff640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf659c640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6ce4003970 0x7f6ce4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf6d9d640 1 -- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6ce4003970 msgr2=0x7f6ce4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf6d9d640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6ce4003970 0x7f6ce4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf6d9d640 1 -- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce40047c0 msgr2=0x7f6ce4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf6d9d640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.391+0000 7f6cf6d9d640 1 -- 172.21.15.31:0/422772001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4003520 con 0x7f6ce4005180 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cf6d9d640 1 -- 172.21.15.31:0/422772001 reap_dead start 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cf6d9d640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ce4005180 0x7f6ce400b980 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000a800 tx=0x7f6ce40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dc18dfe65d2e3f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00072d0 con 0x7f6ce4005180 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf002f070 con 0x7f6ce4005180 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cf5d9b640 1 -- 172.21.15.31:0/422772001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0007670 con 0x7f6ce4005180 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/422772001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ce4005180 msgr2=0x7f6ce400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ce4005180 0x7f6ce400b980 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000a800 tx=0x7f6ce40057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/422772001 shutdown_connections 2024-01-19T04:56:50.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/422772001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ce4005180 0x7f6ce400b980 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/422772001 >> 172.21.15.31:0/422772001 conn(0x7f6cf80d1100 msgr2=0x7f6cf80d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:50.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/422772001 shutdown_connections 2024-01-19T04:56:50.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.392+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/422772001 wait complete. 2024-01-19T04:56:50.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 Processor -- start 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 -- start start 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce4005180 0x7f6cf80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf80b53f0 0x7f6cf814b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6cf80c6ed0 0x7f6cf80c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b5870 con 0x7f6cf80b53f0 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b4350 con 0x7f6ce4005180 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.394+0000 7f6cfcf9e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b4630 con 0x7f6cf80c6ed0 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf80b53f0 0x7f6cf814b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf80b53f0 0x7f6cf814b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39734/0 (socket says 172.21.15.31:39734) 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 -- 172.21.15.31:0/494342169 learned_addr learned my addr 172.21.15.31:0/494342169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6cf659c640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce4005180 0x7f6cf80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6cf6d9d640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6cf80c6ed0 0x7f6cf80c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 -- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6cf80c6ed0 msgr2=0x7f6cf80c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6cf80c6ed0 0x7f6cf80c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 -- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce4005180 msgr2=0x7f6cf80b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce4005180 0x7f6cf80b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 -- 172.21.15.31:0/494342169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80b4910 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6cf659c640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce4005180 0x7f6cf80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceffff640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf80b53f0 0x7f6cf814b6d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000f5f0 tx=0x7f6ce000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceeffd640 1 -- 172.21.15.31:0/494342169 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00052e0 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceeffd640 1 -- 172.21.15.31:0/494342169 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0005440 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6ceeffd640 1 -- 172.21.15.31:0/494342169 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0003810 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf80b4bf0 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.395+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf80c7350 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.396+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf8074900 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.397+0000 7f6ceeffd640 1 -- 172.21.15.31:0/494342169 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6ce0003970 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.397+0000 7f6ceeffd640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6cd806aed0 0x7f6cd806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.397+0000 7f6ceeffd640 1 -- 172.21.15.31:0/494342169 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6ce00895e0 con 0x7f6cf80b53f0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.398+0000 7f6cf659c640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6cd806aed0 0x7f6cd806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.398+0000 7f6cf659c640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6cd806aed0 0x7f6cd806d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006ab0 tx=0x7f6ce8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:50.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.401+0000 7f6ceeffd640 1 -- 172.21.15.31:0/494342169 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6ce00543b0 con 0x7f6cf80b53f0 2024-01-19T04:56:50.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.648+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f6cf80b2780 con 0x7f6cf80b53f0 2024-01-19T04:56:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.650+0000 7f6ceeffd640 1 -- 172.21.15.31:0/494342169 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v11) v1 ==== 211+0+168 (secure 0 0 0) 0x7f6ce00579d0 con 0x7f6cf80b53f0 2024-01-19T04:56:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:56:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:56:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow rwp" 2024-01-19T04:56:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:56:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-01-19T04:56:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:56:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6cd806aed0 msgr2=0x7f6cd806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6cd806aed0 0x7f6cd806d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006ab0 tx=0x7f6ce8008040 comp rx=0 tx=0).stop 2024-01-19T04:56:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf80b53f0 msgr2=0x7f6cf814b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf80b53f0 0x7f6cf814b6d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000f5f0 tx=0x7f6ce000fab0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 shutdown_connections 2024-01-19T04:56:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6cd806aed0 0x7f6cd806d390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6cf80c6ed0 0x7f6cf80c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf80b53f0 0x7f6cf814b6d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 --2- 172.21.15.31:0/494342169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce4005180 0x7f6cf80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 >> 172.21.15.31:0/494342169 conn(0x7f6cf80d1100 msgr2=0x7f6cf815ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 shutdown_connections 2024-01-19T04:56:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.652+0000 7f6cfcf9e640 1 -- 172.21.15.31:0/494342169 wait complete. 2024-01-19T04:56:50.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:56:50.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.764+0000 7f79455be640 1 Processor -- start 2024-01-19T04:56:50.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.764+0000 7f79455be640 1 -- start start 2024-01-19T04:56:50.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f79455be640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f79400cfd10 0x7f79400d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f79455be640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 0x7f79400d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f79455be640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400d06b0 con 0x7f7940160bb0 2024-01-19T04:56:50.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f79455be640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400d18c0 con 0x7f79400cfd10 2024-01-19T04:56:50.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f79455be640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400d1b90 con 0x7f79400d0ef0 2024-01-19T04:56:50.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f793effd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7940160bb0 0x7f79400b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33922/0 (socket says 172.21.15.31:33922) 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f793effd640 1 -- 172.21.15.31:0/2880795754 learned_addr learned my addr 172.21.15.31:0/2880795754 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f793f7fe640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 0x7f79400d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f793e7fc640 1 --2- 172.21.15.31:0/2880795754 >> v2:172.21.15.137:3300/0 conn(0x7f79400cfd10 0x7f79400d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478966095 0 0) 0x7f79400d06b0 con 0x7f7940160bb0 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.765+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f792c003520 con 0x7f7940160bb0 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3958988623 0 0) 0x7f792c003520 con 0x7f7940160bb0 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 msgr2=0x7f79400d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 0x7f79400d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 >> v2:172.21.15.137:3300/0 conn(0x7f79400cfd10 msgr2=0x7f79400d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 --2- 172.21.15.31:0/2880795754 >> v2:172.21.15.137:3300/0 conn(0x7f79400cfd10 0x7f79400d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400d1e70 con 0x7f7940160bb0 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f7938001670 con 0x7f7940160bb0 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793f7fe640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 0x7f79400d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:50.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793e7fc640 1 --2- 172.21.15.31:0/2880795754 >> v2:172.21.15.137:3300/0 conn(0x7f79400cfd10 0x7f79400d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 >> v1:172.21.15.31:6789/0 conn(0x7f7940160bb0 legacy=0x7f79400b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 0x7f792c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c003eb0 0x7f792c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c004d00 0x7f792c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400d18c0 con 0x7f792c004d00 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400d1b90 con 0x7f792c003eb0 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c00b980 con 0x7f79400d0ef0 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793f7fe640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c004d00 0x7f792c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793effd640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 0x7f792c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793e7fc640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c003eb0 0x7f792c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793f7fe640 1 -- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 msgr2=0x7f792c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793f7fe640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400d0ef0 0x7f792c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793f7fe640 1 -- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c003eb0 msgr2=0x7f792c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793f7fe640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c003eb0 0x7f792c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.766+0000 7f793f7fe640 1 -- 172.21.15.31:0/2880795754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7930009a70 con 0x7f792c004d00 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.767+0000 7f793f7fe640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c004d00 0x7f792c005180 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f793000e350 tx=0x7f792c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e29a0bde56ff6 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.767+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793002cd70 con 0x7f792c004d00 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.767+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793000f040 con 0x7f792c004d00 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.767+0000 7f793dffb640 1 -- 172.21.15.31:0/2880795754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930034640 con 0x7f792c004d00 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.767+0000 7f79455be640 1 -- 172.21.15.31:0/2880795754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c004d00 msgr2=0x7f792c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.767+0000 7f79455be640 1 --2- 172.21.15.31:0/2880795754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c004d00 0x7f792c005180 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f793000e350 tx=0x7f792c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.767+0000 7f793f7fe640 1 -- 172.21.15.31:0/2880795754 reap_dead start 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.768+0000 7f79455be640 1 -- 172.21.15.31:0/2880795754 shutdown_connections 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.768+0000 7f79455be640 1 -- 172.21.15.31:0/2880795754 >> 172.21.15.31:0/2880795754 conn(0x7f794009c600 msgr2=0x7f79400d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.768+0000 7f79455be640 1 -- 172.21.15.31:0/2880795754 shutdown_connections 2024-01-19T04:56:50.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.768+0000 7f79455be640 1 -- 172.21.15.31:0/2880795754 wait complete. 2024-01-19T04:56:50.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.769+0000 7f79455be640 1 Processor -- start 2024-01-19T04:56:50.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.769+0000 7f79455be640 1 -- start start 2024-01-19T04:56:50.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79401f2820 0x7f79400c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79401f2c00 0x7f794006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400c8dc0 0x7f794006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400c71c0 con 0x7f79401f2c00 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400c74a0 con 0x7f79401f2820 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400c7780 con 0x7f79400c8dc0 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79401f2c00 0x7f794006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79401f2c00 0x7f794006b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39760/0 (socket says 172.21.15.31:39760) 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 -- 172.21.15.31:0/3843069160 learned_addr learned my addr 172.21.15.31:0/3843069160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793f7fe640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400c8dc0 0x7f794006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793effd640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79401f2820 0x7f79400c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 -- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400c8dc0 msgr2=0x7f794006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400c8dc0 0x7f794006bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 -- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79401f2820 msgr2=0x7f79400c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79401f2820 0x7f79400c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 -- 172.21.15.31:0/3843069160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400c7a60 con 0x7f79401f2c00 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f793e7fc640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79401f2c00 0x7f794006b6e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7928011cd0 tx=0x7f792800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79377fe640 1 -- 172.21.15.31:0/3843069160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800fd20 con 0x7f79401f2c00 2024-01-19T04:56:50.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79377fe640 1 -- 172.21.15.31:0/3843069160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800fea0 con 0x7f79401f2c00 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79400c7d40 con 0x7f79401f2c00 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79377fe640 1 -- 172.21.15.31:0/3843069160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928004260 con 0x7f79401f2c00 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.770+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79401f6560 con 0x7f79401f2c00 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.771+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7940067b50 con 0x7f79401f2c00 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.772+0000 7f79377fe640 1 -- 172.21.15.31:0/3843069160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f79280043c0 con 0x7f79401f2c00 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.772+0000 7f79377fe640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792006aed0 0x7f792006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.772+0000 7f79377fe640 1 -- 172.21.15.31:0/3843069160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7928089fb0 con 0x7f79401f2c00 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.772+0000 7f793effd640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792006aed0 0x7f792006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.773+0000 7f793effd640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792006aed0 0x7f792006d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7938004840 tx=0x7f7938009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:50.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.776+0000 7f79377fe640 1 -- 172.21.15.31:0/3843069160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f79280bd0d0 con 0x7f79401f2c00 2024-01-19T04:56:50.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.993+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79400696e0 con 0x7f79401f2c00 2024-01-19T04:56:50.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.993+0000 7f79377fe640 1 -- 172.21.15.31:0/3843069160 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+82 (secure 0 0 0) 0x7f7928022070 con 0x7f79401f2c00 2024-01-19T04:56:50.993 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-01-19T04:56:50.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792006aed0 msgr2=0x7f792006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792006aed0 0x7f792006d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7938004840 tx=0x7f7938009040 comp rx=0 tx=0).stop 2024-01-19T04:56:50.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79401f2c00 msgr2=0x7f794006b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79401f2c00 0x7f794006b6e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7928011cd0 tx=0x7f792800f610 comp rx=0 tx=0).stop 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 shutdown_connections 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792006aed0 0x7f792006d390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f79400c8dc0 0x7f794006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79401f2c00 0x7f794006b6e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 --2- 172.21.15.31:0/3843069160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79401f2820 0x7f79400c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 >> 172.21.15.31:0/3843069160 conn(0x7f794009c600 msgr2=0x7f794015b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 shutdown_connections 2024-01-19T04:56:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:50.995+0000 7f79455be640 1 -- 172.21.15.31:0/3843069160 wait complete. 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:56:51.006 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:51.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.006 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:56:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:56:51.021 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:56:51.022 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42682sec preferred_lft 42682sec 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c57:fcff:feba:867a/64 scope link 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:56:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:51.024 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::98a9:56ff:fef7:e409/64 scope link 2024-01-19T04:56:51.024 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:51.024 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:56:51.024 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:51.024 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::90b1:87ff:fefd:d7a/64 scope link 2024-01-19T04:56:51.024 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:56:51.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.024 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:56:51.086 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:56:51.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.088 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:56:51.143 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:56:51.143 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:56:51.143 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:56:51.143 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:56:51.143 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:56:51.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.144 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:56:51.144 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:56:51.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:56:51.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:56:51.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:56:51.144 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:56:51.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.238 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:56:51.252 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-19T04:56:51.252 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:56:51.252 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:51.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-01-19T04:56:51.319 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-19T04:56:51.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:56:51.319 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:51.374 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:56:51.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:56:51.374 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:51.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:56:51.496 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:56:51.552 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:56:51.553 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:56:51.554 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:56:51.555 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:56:51.555 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:56:51.555 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:51.555 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:51.555 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:51.555 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:51.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.555 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:56:51.607 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:56:51.608 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-19T04:56:51.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:56:51.677 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:56:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:56:51.734 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:56:51.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.735 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:56:51.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-01-19T04:56:51.736+0000 7f867663a480 -1 init, newargv = 0x55cc8eb543f0 newargc=15 2024-01-19T04:56:51.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[53261]: starting ceph client 2024-01-19T04:56:51.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[53261]: starting fuse 2024-01-19T04:56:51.751 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:56:51.752 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:56:51.752 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:56:51.753 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:56:51.754 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:56:51.754 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-01-19T04:56:51.754 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:56:51.873 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.53261.asok 2024-01-19T04:56:51.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:56:51.883 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53261.asok 2024-01-19T04:56:51.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:51.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53261.asok status 2024-01-19T04:56:52.020 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:56:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:56:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "53261", 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5655, 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 5655 2024-01-19T04:56:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 4275524027 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 4275524027 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.5655 192.168.0.1:0/4275524027", 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/4275524027", 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:56:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 15, 2024-01-19T04:56:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 37, 2024-01-19T04:56:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:56:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:56:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:56:52.023 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:56:52.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:52.029 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:56:52.029 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:56:52.044 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:52.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:52.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:52.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:52.111 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-01-19T04:56:52.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:52.167 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-01-19T04:56:52.226 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:52.226 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:52.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.423+0000 7f047bcca640 1 Processor -- start 2024-01-19T04:56:52.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.423+0000 7f047bcca640 1 -- start start 2024-01-19T04:56:52.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f047bcca640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0474150a50 0x7f0474159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f047bcca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f047bcca640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f04740b2f90 0x7f047415d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f047bcca640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04740cba30 con 0x7f0474150a50 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f047bcca640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04740b3400 con 0x7f04740b2f90 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f047bcca640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0474070e50 con 0x7f04740ce550 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0474150a50 0x7f0474159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0474150a50 0x7f0474159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39768/0 (socket says 172.21.15.31:39768) 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 -- 172.21.15.31:0/1814560232 learned_addr learned my addr 172.21.15.31:0/1814560232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f047a267640 1 --2- 172.21.15.31:0/1814560232 >> v2:172.21.15.137:3300/0 conn(0x7f04740b2f90 0x7f047415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479265640 1 --2- 172.21.15.31:0/1814560232 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 -- 172.21.15.31:0/1814560232 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 msgr2=0x7f04740b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 --2- 172.21.15.31:0/1814560232 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 -- 172.21.15.31:0/1814560232 >> v2:172.21.15.137:3300/0 conn(0x7f04740b2f90 msgr2=0x7f047415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 --2- 172.21.15.31:0/1814560232 >> v2:172.21.15.137:3300/0 conn(0x7f04740b2f90 0x7f047415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.424+0000 7f0479a66640 1 -- 172.21.15.31:0/1814560232 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047415a4f0 con 0x7f0474150a50 2024-01-19T04:56:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.425+0000 7f0479a66640 1 --2- 172.21.15.31:0/1814560232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474150a50 0x7f0474159fb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f047406e1f0 tx=0x7f047002dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585a67434cfac53d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.425+0000 7f0478a64640 1 -- 172.21.15.31:0/1814560232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047002ec50 con 0x7f0474150a50 2024-01-19T04:56:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.425+0000 7f0478a64640 1 -- 172.21.15.31:0/1814560232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0470032070 con 0x7f0474150a50 2024-01-19T04:56:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.425+0000 7f0478a64640 1 -- 172.21.15.31:0/1814560232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470038400 con 0x7f0474150a50 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 -- 172.21.15.31:0/1814560232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474150a50 msgr2=0x7f0474159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 --2- 172.21.15.31:0/1814560232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474150a50 0x7f0474159fb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f047406e1f0 tx=0x7f047002dea0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 -- 172.21.15.31:0/1814560232 shutdown_connections 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 --2- 172.21.15.31:0/1814560232 >> v2:172.21.15.137:3300/0 conn(0x7f04740b2f90 0x7f047415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 --2- 172.21.15.31:0/1814560232 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 --2- 172.21.15.31:0/1814560232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474150a50 0x7f0474159fb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 -- 172.21.15.31:0/1814560232 >> 172.21.15.31:0/1814560232 conn(0x7f047406a790 msgr2=0x7f047406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 -- 172.21.15.31:0/1814560232 shutdown_connections 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.426+0000 7f047bcca640 1 -- 172.21.15.31:0/1814560232 wait complete. 2024-01-19T04:56:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.427+0000 7f047bcca640 1 Processor -- start 2024-01-19T04:56:52.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.427+0000 7f047bcca640 1 -- start start 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f047bcca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740b2f90 0x7f0474140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f047bcca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f047bcca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0474150a50 0x7f04740a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f047bcca640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0474141ba0 con 0x7f04740b2f90 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f047bcca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04740cd580 con 0x7f0474150a50 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f047bcca640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04740cd860 con 0x7f04740ce550 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740b2f90 0x7f0474140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740b2f90 0x7f0474140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39776/0 (socket says 172.21.15.31:39776) 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 -- 172.21.15.31:0/2071522065 learned_addr learned my addr 172.21.15.31:0/2071522065 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479265640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f047a267640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0474150a50 0x7f04740a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 -- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 msgr2=0x7f04740a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 -- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0474150a50 msgr2=0x7f04740a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0474150a50 0x7f04740a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.428+0000 7f0479a66640 1 -- 172.21.15.31:0/2071522065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047000b760 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.429+0000 7f0479a66640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740b2f90 0x7f0474140af0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0470000c00 tx=0x7f04700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.429+0000 7f046a7fc640 1 -- 172.21.15.31:0/2071522065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470006df0 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.429+0000 7f046a7fc640 1 -- 172.21.15.31:0/2071522065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0470043470 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.429+0000 7f046a7fc640 1 -- 172.21.15.31:0/2071522065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470043800 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.429+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0474141550 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.429+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0474141030 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.430+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04740696e0 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.430+0000 7f046a7fc640 1 -- 172.21.15.31:0/2071522065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0470043960 con 0x7f04740b2f90 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.430+0000 7f046a7fc640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f045806aed0 0x7f045806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.431+0000 7f046a7fc640 1 -- 172.21.15.31:0/2071522065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f04700ae470 con 0x7f04740b2f90 2024-01-19T04:56:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.431+0000 7f0479265640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f045806aed0 0x7f045806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.431+0000 7f0479265640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f045806aed0 0x7f045806d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0464006fd0 tx=0x7f0464008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:52.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.435+0000 7f046a7fc640 1 -- 172.21.15.31:0/2071522065 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f04700791b0 con 0x7f04740b2f90 2024-01-19T04:56:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.614+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0474070a00 con 0x7f04740b2f90 2024-01-19T04:56:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.615+0000 7f046a7fc640 1 -- 172.21.15.31:0/2071522065 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f047007c7d0 con 0x7f04740b2f90 2024-01-19T04:56:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:52.615 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:52.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f045806aed0 msgr2=0x7f045806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f045806aed0 0x7f045806d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0464006fd0 tx=0x7f0464008040 comp rx=0 tx=0).stop 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740b2f90 msgr2=0x7f0474140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740b2f90 0x7f0474140af0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0470000c00 tx=0x7f04700046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 shutdown_connections 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f045806aed0 0x7f045806d390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0474150a50 0x7f04740a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04740ce550 0x7f04740a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 --2- 172.21.15.31:0/2071522065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740b2f90 0x7f0474140af0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 >> 172.21.15.31:0/2071522065 conn(0x7f047406a790 msgr2=0x7f04740aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 shutdown_connections 2024-01-19T04:56:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.617+0000 7f047bcca640 1 -- 172.21.15.31:0/2071522065 wait complete. 2024-01-19T04:56:52.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:56:23.785927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:56:24.240600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-01-19T04:56:32.396495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:52.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:52.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.732+0000 7fd97c885640 1 Processor -- start 2024-01-19T04:56:52.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.732+0000 7fd97c885640 1 -- start start 2024-01-19T04:56:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.732+0000 7fd97c885640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd97815d3a0 0x7fd97806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd97c885640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd9780cf570 0x7fd978161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd97c885640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9780cf9f0 con 0x7fd97815d780 2024-01-19T04:56:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd97c885640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9780cfcd0 con 0x7fd9780cf570 2024-01-19T04:56:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd97c885640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780cffb0 con 0x7fd97815d3a0 2024-01-19T04:56:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd97659c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd97815d3a0 0x7fd97806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd975d9b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd9780cf570 0x7fd978161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd975d9b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd9780cf570 0x7fd978161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:37960/0 (socket says 172.21.15.31:37960) 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd975d9b640 1 -- 172.21.15.31:0/3724258052 learned_addr learned my addr 172.21.15.31:0/3724258052 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd976d9d640 1 --1- 172.21.15.31:0/3724258052 >> v1:172.21.15.137:6789/0 conn(0x7fd97815d780 0x7fd9780cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:58078/0 (socket says 172.21.15.31:58078) 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd975d9b640 1 -- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd97815d3a0 msgr2=0x7fd97806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd975d9b640 1 --2- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd97815d3a0 0x7fd97806c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd975d9b640 1 -- 172.21.15.31:0/3724258052 >> v1:172.21.15.137:6789/0 conn(0x7fd97815d780 legacy=0x7fd9780cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd975d9b640 1 -- 172.21.15.31:0/3724258052 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780d0290 con 0x7fd9780cf570 2024-01-19T04:56:52.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.733+0000 7fd97659c640 1 --2- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd97815d3a0 0x7fd97806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.734+0000 7fd975d9b640 1 --2- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9780cf570 0x7fd978161150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009870 tx=0x7fd96c02b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71121eb974f78a59 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:52.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.734+0000 7fd97559a640 1 -- 172.21.15.31:0/3724258052 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c004230 con 0x7fd9780cf570 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.734+0000 7fd97559a640 1 -- 172.21.15.31:0/3724258052 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c02e050 con 0x7fd9780cf570 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.734+0000 7fd97559a640 1 -- 172.21.15.31:0/3724258052 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c003610 con 0x7fd9780cf570 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 -- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9780cf570 msgr2=0x7fd978161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 --2- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9780cf570 0x7fd978161150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009870 tx=0x7fd96c02b7e0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 -- 172.21.15.31:0/3724258052 shutdown_connections 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 --2- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9780cf570 0x7fd978161150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 --2- 172.21.15.31:0/3724258052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd97815d3a0 0x7fd97806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 -- 172.21.15.31:0/3724258052 >> 172.21.15.31:0/3724258052 conn(0x7fd9780d1100 msgr2=0x7fd9780d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 -- 172.21.15.31:0/3724258052 shutdown_connections 2024-01-19T04:56:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.735+0000 7fd97c885640 1 -- 172.21.15.31:0/3724258052 wait complete. 2024-01-19T04:56:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.736+0000 7fd97c885640 1 Processor -- start 2024-01-19T04:56:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.736+0000 7fd97c885640 1 -- start start 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd97c885640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd9780cf570 0x7fd9780c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd97c885640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd97815d3a0 0x7fd9780c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd97c885640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97815d780 0x7fd9780b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd97c885640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd978152950 con 0x7fd97815d780 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd97c885640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9781f4570 con 0x7fd97815d3a0 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd97c885640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780c66e0 con 0x7fd9780cf570 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97815d780 0x7fd9780b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97815d780 0x7fd9780b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39792/0 (socket says 172.21.15.31:39792) 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 -- 172.21.15.31:0/1686364964 learned_addr learned my addr 172.21.15.31:0/1686364964 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd97659c640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd9780cf570 0x7fd9780c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd975d9b640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd97815d3a0 0x7fd9780c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 -- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd9780cf570 msgr2=0x7fd9780c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd9780cf570 0x7fd9780c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 -- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd97815d3a0 msgr2=0x7fd9780c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd97815d3a0 0x7fd9780c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd976d9d640 1 -- 172.21.15.31:0/1686364964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9680093a0 con 0x7fd97815d780 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.737+0000 7fd975d9b640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd97815d3a0 0x7fd9780c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.738+0000 7fd976d9d640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97815d780 0x7fd9780b5e50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd97000c910 tx=0x7fd97000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.738+0000 7fd966ffd640 1 -- 172.21.15.31:0/1686364964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97000a700 con 0x7fd97815d780 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.738+0000 7fd966ffd640 1 -- 172.21.15.31:0/1686364964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97000a880 con 0x7fd97815d780 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.738+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c0094d0 con 0x7fd97815d780 2024-01-19T04:56:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.738+0000 7fd966ffd640 1 -- 172.21.15.31:0/1686364964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97000ac70 con 0x7fd97815d780 2024-01-19T04:56:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.738+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9781f3cd0 con 0x7fd97815d780 2024-01-19T04:56:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.739+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd978074900 con 0x7fd97815d780 2024-01-19T04:56:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.739+0000 7fd966ffd640 1 -- 172.21.15.31:0/1686364964 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd9700070d0 con 0x7fd97815d780 2024-01-19T04:56:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.739+0000 7fd966ffd640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd95806aed0 0x7fd95806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.740+0000 7fd966ffd640 1 -- 172.21.15.31:0/1686364964 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd970088e00 con 0x7fd97815d780 2024-01-19T04:56:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.740+0000 7fd97659c640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd95806aed0 0x7fd95806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.740+0000 7fd97659c640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd95806aed0 0x7fd95806d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd968009cc0 tx=0x7fd968009110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:52.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.744+0000 7fd966ffd640 1 -- 172.21.15.31:0/1686364964 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd970053b70 con 0x7fd97815d780 2024-01-19T04:56:52.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.923+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd978067030 con 0x7fd97815d780 2024-01-19T04:56:52.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.923+0000 7fd966ffd640 1 -- 172.21.15.31:0/1686364964 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9700180a0 con 0x7fd97815d780 2024-01-19T04:56:52.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:52.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd95806aed0 msgr2=0x7fd95806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd95806aed0 0x7fd95806d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd968009cc0 tx=0x7fd968009110 comp rx=0 tx=0).stop 2024-01-19T04:56:52.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97815d780 msgr2=0x7fd9780b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:52.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97815d780 0x7fd9780b5e50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd97000c910 tx=0x7fd97000cdd0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 shutdown_connections 2024-01-19T04:56:52.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd95806aed0 0x7fd95806d390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97815d780 0x7fd9780b5e50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd97815d3a0 0x7fd9780c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 --2- 172.21.15.31:0/1686364964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd9780cf570 0x7fd9780c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 >> 172.21.15.31:0/1686364964 conn(0x7fd9780d1100 msgr2=0x7fd9780d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 shutdown_connections 2024-01-19T04:56:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:52.925+0000 7fd97c885640 1 -- 172.21.15.31:0/1686364964 wait complete. 2024-01-19T04:56:52.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.040+0000 7f1268501640 1 Processor -- start 2024-01-19T04:56:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.040+0000 7f1268501640 1 -- start start 2024-01-19T04:56:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1268501640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1260143440 0x7f12600cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1268501640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f126006ddc0 0x7f12600b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1268501640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126015ecb0 con 0x7f12600b33f0 2024-01-19T04:56:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1268501640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1260072440 con 0x7f126006ddc0 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1268501640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1260143820 con 0x7f1260143440 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1266a9e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f126006ddc0 0x7f12600b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1265a9c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f12600b33f0 0x7f126006d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33926/0 (socket says 172.21.15.31:33926) 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1265a9c640 1 -- 172.21.15.31:0/50840201 learned_addr learned my addr 172.21.15.31:0/50840201 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f126629d640 1 --2- 172.21.15.31:0/50840201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1260143440 0x7f12600cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1266a9e640 1 -- 172.21.15.31:0/50840201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1260143440 msgr2=0x7f12600cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1266a9e640 1 --2- 172.21.15.31:0/50840201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1260143440 0x7f12600cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f1266a9e640 1 -- 172.21.15.31:0/50840201 >> v1:172.21.15.31:6789/0 conn(0x7f12600b33f0 legacy=0x7f126006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.042+0000 7f1266a9e640 1 -- 172.21.15.31:0/50840201 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126015e130 con 0x7f126006ddc0 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.041+0000 7f126529b640 1 -- 172.21.15.31:0/50840201 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213853858 0 0) 0x7f126015ecb0 con 0x7f12600b33f0 2024-01-19T04:56:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.042+0000 7f1266a9e640 1 --2- 172.21.15.31:0/50840201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126006ddc0 0x7f12600b21b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f125c0097d0 tx=0x7f125c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3979b665e86291 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.042+0000 7f126529b640 1 -- 172.21.15.31:0/50840201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c031770 con 0x7f126006ddc0 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.042+0000 7f126529b640 1 -- 172.21.15.31:0/50840201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c0318d0 con 0x7f126006ddc0 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.042+0000 7f126529b640 1 -- 172.21.15.31:0/50840201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c031c60 con 0x7f126006ddc0 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 -- 172.21.15.31:0/50840201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126006ddc0 msgr2=0x7f12600b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 --2- 172.21.15.31:0/50840201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126006ddc0 0x7f12600b21b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f125c0097d0 tx=0x7f125c02b810 comp rx=0 tx=0).stop 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 -- 172.21.15.31:0/50840201 shutdown_connections 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 --2- 172.21.15.31:0/50840201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126006ddc0 0x7f12600b21b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 --2- 172.21.15.31:0/50840201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1260143440 0x7f12600cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 -- 172.21.15.31:0/50840201 >> 172.21.15.31:0/50840201 conn(0x7f126006aa00 msgr2=0x7f126006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 -- 172.21.15.31:0/50840201 shutdown_connections 2024-01-19T04:56:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.043+0000 7f1268501640 1 -- 172.21.15.31:0/50840201 wait complete. 2024-01-19T04:56:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.044+0000 7f1268501640 1 Processor -- start 2024-01-19T04:56:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.044+0000 7f1268501640 1 -- start start 2024-01-19T04:56:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1268501640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f126006ddc0 0x7f126013c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1268501640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12600b33f0 0x7f1260157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1268501640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1260143440 0x7f12601f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1268501640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1260158230 con 0x7f1260143440 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1268501640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1260157c70 con 0x7f12600b33f0 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1268501640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1260157f50 con 0x7f126006ddc0 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1260143440 0x7f12601f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1260143440 0x7f12601f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39812/0 (socket says 172.21.15.31:39812) 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 -- 172.21.15.31:0/4034242900 learned_addr learned my addr 172.21.15.31:0/4034242900 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f126629d640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f126006ddc0 0x7f126013c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1265a9c640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12600b33f0 0x7f1260157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 -- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f126006ddc0 msgr2=0x7f126013c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f126006ddc0 0x7f126013c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 -- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12600b33f0 msgr2=0x7f1260157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12600b33f0 0x7f1260157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1266a9e640 1 -- 172.21.15.31:0/4034242900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c009400 con 0x7f1260143440 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f1265a9c640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12600b33f0 0x7f1260157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.045+0000 7f126629d640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f126006ddc0 0x7f126013c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.046+0000 7f1266a9e640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1260143440 0x7f12601f0e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f125c009720 tx=0x7f125c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.046+0000 7f1256ffd640 1 -- 172.21.15.31:0/4034242900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c032810 con 0x7f1260143440 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.046+0000 7f1256ffd640 1 -- 172.21.15.31:0/4034242900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c032990 con 0x7f1260143440 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.046+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126015b910 con 0x7f1260143440 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.046+0000 7f1256ffd640 1 -- 172.21.15.31:0/4034242900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c032d40 con 0x7f1260143440 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.046+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12601f1fc0 con 0x7f1260143440 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.047+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12600cc0d0 con 0x7f1260143440 2024-01-19T04:56:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.047+0000 7f1256ffd640 1 -- 172.21.15.31:0/4034242900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f125c04d3f0 con 0x7f1260143440 2024-01-19T04:56:53.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.048+0000 7f1256ffd640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f124406aed0 0x7f124406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.048+0000 7f1256ffd640 1 -- 172.21.15.31:0/4034242900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f125c0ab1d0 con 0x7f1260143440 2024-01-19T04:56:53.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.048+0000 7f126629d640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f124406aed0 0x7f124406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.048+0000 7f126629d640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f124406aed0 0x7f124406d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1258009fd0 tx=0x7f125800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.052+0000 7f1256ffd640 1 -- 172.21.15.31:0/4034242900 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f125c075fa0 con 0x7f1260143440 2024-01-19T04:56:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.228+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f126015be30 con 0x7f1260143440 2024-01-19T04:56:53.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.229+0000 7f1256ffd640 1 -- 172.21.15.31:0/4034242900 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f126015be30 con 0x7f1260143440 2024-01-19T04:56:53.229 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f124406aed0 msgr2=0x7f124406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f124406aed0 0x7f124406d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1258009fd0 tx=0x7f125800b040 comp rx=0 tx=0).stop 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1260143440 msgr2=0x7f12601f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1260143440 0x7f12601f0e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f125c009720 tx=0x7f125c0046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 shutdown_connections 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f124406aed0 0x7f124406d390 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1260143440 0x7f12601f0e10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12600b33f0 0x7f1260157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 --2- 172.21.15.31:0/4034242900 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f126006ddc0 0x7f126013c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 >> 172.21.15.31:0/4034242900 conn(0x7f126006aa00 msgr2=0x7f12600cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 shutdown_connections 2024-01-19T04:56:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.231+0000 7f1268501640 1 -- 172.21.15.31:0/4034242900 wait complete. 2024-01-19T04:56:53.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:53.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.346+0000 7f6b33d2c640 1 Processor -- start 2024-01-19T04:56:53.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.346+0000 7f6b33d2c640 1 -- start start 2024-01-19T04:56:53.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b33d2c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6b2c0d18e0 0x7f6b2c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b33d2c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c15dee0 con 0x7f6b2c0d3e50 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b33d2c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c15e1c0 con 0x7f6b2c0d0ef0 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b33d2c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c15e4a0 con 0x7f6b2c0d18e0 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b312c7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6b2c0d3e50 0x7f6b2c0d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33928/0 (socket says 172.21.15.31:33928) 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b312c7640 1 -- 172.21.15.31:0/2563813918 learned_addr learned my addr 172.21.15.31:0/2563813918 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/2563813918 >> v2:172.21.15.137:3300/0 conn(0x7f6b2c0d18e0 0x7f6b2c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758572415 0 0) 0x7f6b2c15dee0 con 0x7f6b2c0d3e50 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.347+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b14003520 con 0x7f6b2c0d3e50 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2942815254 0 0) 0x7f6b14003520 con 0x7f6b2c0d3e50 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 >> v2:172.21.15.137:3300/0 conn(0x7f6b2c0d18e0 msgr2=0x7f6b2c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 --2- 172.21.15.31:0/2563813918 >> v2:172.21.15.137:3300/0 conn(0x7f6b2c0d18e0 0x7f6b2c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 >> v1:172.21.15.137:6789/0 conn(0x7f6b2c0d0ef0 legacy=0x7f6b2c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c15e780 con 0x7f6b2c0d3e50 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f6b20001670 con 0x7f6b2c0d3e50 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 >> v1:172.21.15.31:6789/0 conn(0x7f6b2c0d3e50 legacy=0x7f6b2c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b140047c0 0x7f6b14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b14005180 0x7f6b1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c15e4a0 con 0x7f6b14003970 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1400c020 con 0x7f6b140047c0 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1400c1e0 con 0x7f6b14005180 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b312c7640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b140047c0 0x7f6b14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b322c9640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b14005180 0x7f6b1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b31ac8640 1 -- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b14005180 msgr2=0x7f6b1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b14005180 0x7f6b1400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b31ac8640 1 -- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b140047c0 msgr2=0x7f6b14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b140047c0 0x7f6b14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b31ac8640 1 -- 172.21.15.31:0/2563813918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b14003520 con 0x7f6b14003970 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.348+0000 7f6b322c9640 1 -- 172.21.15.31:0/2563813918 reap_dead start 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.349+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b14003d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009ac0 tx=0x7f6b140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e536e34e58325bcc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.349+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2402c6b0 con 0x7f6b14003970 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.349+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2402c810 con 0x7f6b14003970 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.349+0000 7f6b30ac6640 1 -- 172.21.15.31:0/2563813918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2402cba0 con 0x7f6b14003970 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.350+0000 7f6b33d2c640 1 -- 172.21.15.31:0/2563813918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 msgr2=0x7f6b14003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.350+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b14003d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009ac0 tx=0x7f6b140057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.350+0000 7f6b33d2c640 1 -- 172.21.15.31:0/2563813918 shutdown_connections 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.350+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/2563813918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b14003d90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.350+0000 7f6b33d2c640 1 -- 172.21.15.31:0/2563813918 >> 172.21.15.31:0/2563813918 conn(0x7f6b2c09c600 msgr2=0x7f6b2c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:53.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.350+0000 7f6b33d2c640 1 -- 172.21.15.31:0/2563813918 shutdown_connections 2024-01-19T04:56:53.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.350+0000 7f6b33d2c640 1 -- 172.21.15.31:0/2563813918 wait complete. 2024-01-19T04:56:53.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.351+0000 7f6b33d2c640 1 Processor -- start 2024-01-19T04:56:53.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.351+0000 7f6b33d2c640 1 -- start start 2024-01-19T04:56:53.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b2c154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b2c0b0ab0 0x7f6b2c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c0b0f30 0x7f6b2c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c155030 con 0x7f6b14003970 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c94d0 con 0x7f6b2c0b0f30 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c8430 con 0x7f6b2c0b0ab0 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b2c154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b2c154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39830/0 (socket says 172.21.15.31:39830) 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 -- 172.21.15.31:0/578238424 learned_addr learned my addr 172.21.15.31:0/578238424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b312c7640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b2c0b0ab0 0x7f6b2c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b322c9640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c0b0f30 0x7f6b2c06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 -- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b2c0b0ab0 msgr2=0x7f6b2c06c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b2c0b0ab0 0x7f6b2c06c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 -- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c0b0f30 msgr2=0x7f6b2c06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c0b0f30 0x7f6b2c06cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 -- 172.21.15.31:0/578238424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0c8710 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b312c7640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b2c0b0ab0 0x7f6b2c06c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b31ac8640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b2c154af0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009ac0 tx=0x7f6b24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b2a7fc640 1 -- 172.21.15.31:0/578238424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24003810 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b2a7fc640 1 -- 172.21.15.31:0/578238424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24003970 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c0c89f0 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b2a7fc640 1 -- 172.21.15.31:0/578238424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24003d00 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.352+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c0c8fb0 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.353+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c074900 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.354+0000 7f6b2a7fc640 1 -- 172.21.15.31:0/578238424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6b24040450 con 0x7f6b14003970 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.354+0000 7f6b2a7fc640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b1806af00 0x7f6b1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.354+0000 7f6b2a7fc640 1 -- 172.21.15.31:0/578238424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6b240ab5d0 con 0x7f6b14003970 2024-01-19T04:56:53.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.354+0000 7f6b312c7640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b1806af00 0x7f6b1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.355+0000 7f6b312c7640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b1806af00 0x7f6b1806d3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b20001050 tx=0x7f6b20009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.358+0000 7f6b2a7fc640 1 -- 172.21.15.31:0/578238424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6b240763a0 con 0x7f6b14003970 2024-01-19T04:56:53.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.537+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b2c0d09a0 con 0x7f6b14003970 2024-01-19T04:56:53.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.537+0000 7f6b2a7fc640 1 -- 172.21.15.31:0/578238424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b240044e0 con 0x7f6b14003970 2024-01-19T04:56:53.537 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b1806af00 msgr2=0x7f6b1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b1806af00 0x7f6b1806d3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b20001050 tx=0x7f6b20009040 comp rx=0 tx=0).stop 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 msgr2=0x7f6b2c154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b2c154af0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009ac0 tx=0x7f6b24004730 comp rx=0 tx=0).stop 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 shutdown_connections 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b1806af00 0x7f6b1806d3c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c0b0f30 0x7f6b2c06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6b2c0b0ab0 0x7f6b2c06c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 --2- 172.21.15.31:0/578238424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b14003970 0x7f6b2c154af0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 >> 172.21.15.31:0/578238424 conn(0x7f6b2c09c600 msgr2=0x7f6b2c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:53.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 shutdown_connections 2024-01-19T04:56:53.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.539+0000 7f6b33d2c640 1 -- 172.21.15.31:0/578238424 wait complete. 2024-01-19T04:56:53.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-19T04:56:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.654+0000 7f5983261640 1 Processor -- start 2024-01-19T04:56:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.654+0000 7f5983261640 1 -- start start 2024-01-19T04:56:53.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f5983261640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f597c143be0 0x7f597c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f5983261640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 0x7f597c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f5983261640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597c0b2f50 con 0x7f597c06f1f0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f5983261640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597c143fc0 con 0x7f597c143be0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f5983261640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c06f680 con 0x7f597c06d6b0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f59817fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f597c06f1f0 0x7f597c15dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33942/0 (socket says 172.21.15.31:33942) 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f59817fe640 1 -- 172.21.15.31:0/2738840176 learned_addr learned my addr 172.21.15.31:0/2738840176 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597bfff640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 0x7f597c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f5980ffd640 1 --2- 172.21.15.31:0/2738840176 >> v2:172.21.15.137:3300/0 conn(0x7f597c143be0 0x7f597c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83485958 0 0) 0x7f597c0b2f50 con 0x7f597c06f1f0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f596c003520 con 0x7f597c06f1f0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3386465370 0 0) 0x7f596c003520 con 0x7f597c06f1f0 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 msgr2=0x7f597c06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 0x7f597c06ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 >> v2:172.21.15.137:3300/0 conn(0x7f597c143be0 msgr2=0x7f597c06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 --2- 172.21.15.31:0/2738840176 >> v2:172.21.15.137:3300/0 conn(0x7f597c143be0 0x7f597c06e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c0ce6f0 con 0x7f597c06f1f0 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f5980ffd640 1 --2- 172.21.15.31:0/2738840176 >> v2:172.21.15.137:3300/0 conn(0x7f597c143be0 0x7f597c06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f59740019f0 con 0x7f597c06f1f0 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.655+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 >> v1:172.21.15.31:6789/0 conn(0x7f597c06f1f0 legacy=0x7f597c15dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597b7fe640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f596c003970 0x7f596c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597b7fe640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f596c0047c0 0x7f596c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597b7fe640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 0x7f596c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c143fc0 con 0x7f596c0047c0 2024-01-19T04:56:53.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c06f680 con 0x7f596c003970 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c00b980 con 0x7f597c06d6b0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597bfff640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f596c0047c0 0x7f596c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f5980ffd640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f596c003970 0x7f596c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f59817fe640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 0x7f596c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597bfff640 1 -- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 msgr2=0x7f596c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597bfff640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f597c06d6b0 0x7f596c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597bfff640 1 -- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f596c003970 msgr2=0x7f596c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597bfff640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f596c003970 0x7f596c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.656+0000 7f597bfff640 1 -- 172.21.15.31:0/2738840176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c003520 con 0x7f596c0047c0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f597bfff640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f596c0047c0 0x7f596c004c40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5968009ac0 tx=0x7f596c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b944310b72f2efc server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596802c6b0 con 0x7f596c0047c0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596802c810 con 0x7f596c0047c0 2024-01-19T04:56:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f597b7fe640 1 -- 172.21.15.31:0/2738840176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596802cba0 con 0x7f596c0047c0 2024-01-19T04:56:53.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f5983261640 1 -- 172.21.15.31:0/2738840176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f596c0047c0 msgr2=0x7f596c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f5983261640 1 --2- 172.21.15.31:0/2738840176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f596c0047c0 0x7f596c004c40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5968009ac0 tx=0x7f596c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f59817fe640 1 -- 172.21.15.31:0/2738840176 reap_dead start 2024-01-19T04:56:53.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f5983261640 1 -- 172.21.15.31:0/2738840176 shutdown_connections 2024-01-19T04:56:53.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.657+0000 7f5983261640 1 -- 172.21.15.31:0/2738840176 >> 172.21.15.31:0/2738840176 conn(0x7f597c0b1c50 msgr2=0x7f597c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:53.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.658+0000 7f5983261640 1 -- 172.21.15.31:0/2738840176 shutdown_connections 2024-01-19T04:56:53.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.658+0000 7f5983261640 1 -- 172.21.15.31:0/2738840176 wait complete. 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.659+0000 7f5983261640 1 Processor -- start 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.659+0000 7f5983261640 1 -- start start 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f596c0047c0 0x7f597c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c1f3dd0 0x7f597c13a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f597c13a860 0x7f597c13b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c15b130 con 0x7f597c1f3dd0 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c13adc0 con 0x7f597c13a860 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c136430 con 0x7f596c0047c0 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c1f3dd0 0x7f597c13a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c1f3dd0 0x7f597c13a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39856/0 (socket says 172.21.15.31:39856) 2024-01-19T04:56:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 -- 172.21.15.31:0/3890385223 learned_addr learned my addr 172.21.15.31:0/3890385223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f59817fe640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f597c13a860 0x7f597c13b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5980ffd640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f596c0047c0 0x7f597c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 -- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f596c0047c0 msgr2=0x7f597c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f596c0047c0 0x7f597c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 -- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f597c13a860 msgr2=0x7f597c13b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f597c13a860 0x7f597c13b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 -- 172.21.15.31:0/3890385223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c136710 con 0x7f597c1f3dd0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f597bfff640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c1f3dd0 0x7f597c13a320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f5968004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f59797fa640 1 -- 172.21.15.31:0/3890385223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968003810 con 0x7f597c1f3dd0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f59797fa640 1 -- 172.21.15.31:0/3890385223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968003970 con 0x7f597c1f3dd0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597c1369f0 con 0x7f597c1f3dd0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f59797fa640 1 -- 172.21.15.31:0/3890385223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968003d00 con 0x7f597c1f3dd0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.660+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c138da0 con 0x7f597c1f3dd0 2024-01-19T04:56:53.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.661+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597c074900 con 0x7f597c1f3dd0 2024-01-19T04:56:53.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.662+0000 7f59797fa640 1 -- 172.21.15.31:0/3890385223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5968047450 con 0x7f597c1f3dd0 2024-01-19T04:56:53.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.662+0000 7f59797fa640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f596006af00 0x7f596006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.662+0000 7f59797fa640 1 -- 172.21.15.31:0/3890385223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f59680ab930 con 0x7f597c1f3dd0 2024-01-19T04:56:53.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.662+0000 7f5980ffd640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f596006af00 0x7f596006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.663+0000 7f5980ffd640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f596006af00 0x7f596006d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f597c13d2c0 tx=0x7f5970008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.666+0000 7f59797fa640 1 -- 172.21.15.31:0/3890385223 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f59680767b0 con 0x7f597c1f3dd0 2024-01-19T04:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.843+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f597c15b130 con 0x7f597c1f3dd0 2024-01-19T04:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.844+0000 7f59797fa640 1 -- 172.21.15.31:0/3890385223 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7f597c15b130 con 0x7f597c1f3dd0 2024-01-19T04:56:53.843 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.845+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f596006af00 msgr2=0x7f596006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.845+0000 7f5983261640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f596006af00 0x7f596006d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f597c13d2c0 tx=0x7f5970008040 comp rx=0 tx=0).stop 2024-01-19T04:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.845+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c1f3dd0 msgr2=0x7f597c13a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.845+0000 7f5983261640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c1f3dd0 0x7f597c13a320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f5968004730 comp rx=0 tx=0).stop 2024-01-19T04:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 shutdown_connections 2024-01-19T04:56:53.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f596006af00 0x7f596006d3c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f597c13a860 0x7f597c13b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c1f3dd0 0x7f597c13a320 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 --2- 172.21.15.31:0/3890385223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f596c0047c0 0x7f597c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 >> 172.21.15.31:0/3890385223 conn(0x7f597c0b1c50 msgr2=0x7f597c0cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:53.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 shutdown_connections 2024-01-19T04:56:53.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.846+0000 7f5983261640 1 -- 172.21.15.31:0/3890385223 wait complete. 2024-01-19T04:56:53.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.958+0000 7fe2477fc640 1 Processor -- start 2024-01-19T04:56:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.958+0000 7fe2477fc640 1 -- start start 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe2477fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2400ceff0 0x7fe24015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe2477fc640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400d1600 con 0x7fe2400d18e0 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe2477fc640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24015dee0 con 0x7fe2400d3e50 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe2477fc640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400cf460 con 0x7fe2400ceff0 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe245d99640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2400ceff0 0x7fe24015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe245598640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe2400d18e0 0x7fe2400d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33948/0 (socket says 172.21.15.31:33948) 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe245598640 1 -- 172.21.15.31:0/1071137299 learned_addr learned my addr 172.21.15.31:0/1071137299 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe245d99640 1 -- 172.21.15.31:0/1071137299 >> v1:172.21.15.137:6789/0 conn(0x7fe2400d3e50 legacy=0x7fe2400d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe245d99640 1 -- 172.21.15.31:0/1071137299 >> v1:172.21.15.31:6789/0 conn(0x7fe2400d18e0 legacy=0x7fe2400d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.959+0000 7fe245d99640 1 -- 172.21.15.31:0/1071137299 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24015e1c0 con 0x7fe2400ceff0 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.960+0000 7fe245d99640 1 --2- 172.21.15.31:0/1071137299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400ceff0 0x7fe24015d860 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe230009d90 tx=0x7fe23002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a669092bddb239 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.960+0000 7fe23ffff640 1 -- 172.21.15.31:0/1071137299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23002cc60 con 0x7fe2400ceff0 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.960+0000 7fe23ffff640 1 -- 172.21.15.31:0/1071137299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23002cdc0 con 0x7fe2400ceff0 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.960+0000 7fe23ffff640 1 -- 172.21.15.31:0/1071137299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230034660 con 0x7fe2400ceff0 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.961+0000 7fe2477fc640 1 -- 172.21.15.31:0/1071137299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400ceff0 msgr2=0x7fe24015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.961+0000 7fe2477fc640 1 --2- 172.21.15.31:0/1071137299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400ceff0 0x7fe24015d860 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe230009d90 tx=0x7fe23002beb0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.961+0000 7fe2477fc640 1 -- 172.21.15.31:0/1071137299 shutdown_connections 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.961+0000 7fe2477fc640 1 --2- 172.21.15.31:0/1071137299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400ceff0 0x7fe24015d860 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.961+0000 7fe2477fc640 1 -- 172.21.15.31:0/1071137299 >> 172.21.15.31:0/1071137299 conn(0x7fe24009c600 msgr2=0x7fe2400d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.961+0000 7fe2477fc640 1 -- 172.21.15.31:0/1071137299 shutdown_connections 2024-01-19T04:56:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.961+0000 7fe2477fc640 1 -- 172.21.15.31:0/1071137299 wait complete. 2024-01-19T04:56:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.962+0000 7fe2477fc640 1 Processor -- start 2024-01-19T04:56:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.962+0000 7fe2477fc640 1 -- start start 2024-01-19T04:56:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe2477fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2400ceff0 0x7fe240155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe2477fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2400d18e0 0x7fe2400cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe2477fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400d3e50 0x7fe2400b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe2477fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240151010 con 0x7fe2400d3e50 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe2477fc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400cd590 con 0x7fe2400ceff0 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe2477fc640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400cd2b0 con 0x7fe2400d18e0 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400d3e50 0x7fe2400b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400d3e50 0x7fe2400b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39866/0 (socket says 172.21.15.31:39866) 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 -- 172.21.15.31:0/746332469 learned_addr learned my addr 172.21.15.31:0/746332469 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245598640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2400ceff0 0x7fe240155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe244d97640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2400d18e0 0x7fe2400cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 -- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2400d18e0 msgr2=0x7fe2400cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2400d18e0 0x7fe2400cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 -- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2400ceff0 msgr2=0x7fe240155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2400ceff0 0x7fe240155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 -- 172.21.15.31:0/746332469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230009a70 con 0x7fe2400d3e50 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245598640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2400ceff0 0x7fe240155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe245d99640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400d3e50 0x7fe2400b0c50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe230009d90 tx=0x7fe230004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe23dffb640 1 -- 172.21.15.31:0/746332469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230040450 con 0x7fe2400d3e50 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe23dffb640 1 -- 172.21.15.31:0/746332469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2300405d0 con 0x7fe2400d3e50 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe240150950 con 0x7fe2400d3e50 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.963+0000 7fe23dffb640 1 -- 172.21.15.31:0/746332469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230040980 con 0x7fe2400d3e50 2024-01-19T04:56:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.964+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe240156260 con 0x7fe2400d3e50 2024-01-19T04:56:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.965+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240151150 con 0x7fe2400d3e50 2024-01-19T04:56:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.965+0000 7fe23dffb640 1 -- 172.21.15.31:0/746332469 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe230040ae0 con 0x7fe2400d3e50 2024-01-19T04:56:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.965+0000 7fe23dffb640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe22c06aed0 0x7fe22c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.966+0000 7fe23dffb640 1 -- 172.21.15.31:0/746332469 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe2300ac340 con 0x7fe2400d3e50 2024-01-19T04:56:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.966+0000 7fe245598640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe22c06aed0 0x7fe22c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.966+0000 7fe245598640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe22c06aed0 0x7fe22c06d390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe238000f60 tx=0x7fe238009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:53.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:53.970+0000 7fe23dffb640 1 -- 172.21.15.31:0/746332469 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe240151150 con 0x7fe2400d3e50 2024-01-19T04:56:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.161+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe240151150 con 0x7fe2400d3e50 2024-01-19T04:56:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.162+0000 7fe23dffb640 1 -- 172.21.15.31:0/746332469 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe240151150 con 0x7fe2400d3e50 2024-01-19T04:56:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.163+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe22c06aed0 msgr2=0x7fe22c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.163+0000 7fe2477fc640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe22c06aed0 0x7fe22c06d390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe238000f60 tx=0x7fe238009040 comp rx=0 tx=0).stop 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.163+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400d3e50 msgr2=0x7fe2400b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.163+0000 7fe2477fc640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400d3e50 0x7fe2400b0c50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe230009d90 tx=0x7fe230004a40 comp rx=0 tx=0).stop 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 shutdown_connections 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe22c06aed0 0x7fe22c06d390 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2400d3e50 0x7fe2400b0c50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2400d18e0 0x7fe2400cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 --2- 172.21.15.31:0/746332469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2400ceff0 0x7fe240155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 >> 172.21.15.31:0/746332469 conn(0x7fe24009c600 msgr2=0x7fe2400b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 shutdown_connections 2024-01-19T04:56:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.164+0000 7fe2477fc640 1 -- 172.21.15.31:0/746332469 wait complete. 2024-01-19T04:56:54.174 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:54.174 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:54.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:54.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.329+0000 7f308237d640 1 Processor -- start 2024-01-19T04:56:54.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.329+0000 7f308237d640 1 -- start start 2024-01-19T04:56:54.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f308237d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f307c160bb0 0x7f307c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f308237d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0d1600 0x7f307c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f308237d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0d06b0 con 0x7f307c0cfd10 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f308237d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0b66b0 con 0x7f307c160bb0 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f308237d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0d1b90 con 0x7f307c0d1600 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f307b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f307c0cfd10 0x7f307c0d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33956/0 (socket says 172.21.15.31:33956) 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f307b7fe640 1 -- 172.21.15.31:0/4054288790 learned_addr learned my addr 172.21.15.31:0/4054288790 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f307bfff640 1 --2- 172.21.15.31:0/4054288790 >> v2:172.21.15.137:3300/0 conn(0x7f307c160bb0 0x7f307c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.330+0000 7f308091a640 1 --2- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0d1600 0x7f307c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.331+0000 7f307affd640 1 -- 172.21.15.31:0/4054288790 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647869123 0 0) 0x7f307c0d06b0 con 0x7f307c0cfd10 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.331+0000 7f307affd640 1 -- 172.21.15.31:0/4054288790 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3068003520 con 0x7f307c0cfd10 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.331+0000 7f307bfff640 1 -- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0d1600 msgr2=0x7f307c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.331+0000 7f307bfff640 1 --2- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0d1600 0x7f307c0d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.331+0000 7f307bfff640 1 -- 172.21.15.31:0/4054288790 >> v1:172.21.15.31:6789/0 conn(0x7f307c0cfd10 legacy=0x7f307c0d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.331+0000 7f307bfff640 1 -- 172.21.15.31:0/4054288790 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0d1e70 con 0x7f307c160bb0 2024-01-19T04:56:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.331+0000 7f308091a640 1 --2- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0d1600 0x7f307c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f307bfff640 1 --2- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c160bb0 0x7f307c0b6170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f30640097a0 tx=0x7f306402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=334e6581f29b9581 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f307affd640 1 -- 172.21.15.31:0/4054288790 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064031830 con 0x7f307c160bb0 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f307affd640 1 -- 172.21.15.31:0/4054288790 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064031990 con 0x7f307c160bb0 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f307affd640 1 -- 172.21.15.31:0/4054288790 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064031d00 con 0x7f307c160bb0 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 -- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c160bb0 msgr2=0x7f307c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 --2- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c160bb0 0x7f307c0b6170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f30640097a0 tx=0x7f306402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 -- 172.21.15.31:0/4054288790 shutdown_connections 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 --2- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0d1600 0x7f307c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 --2- 172.21.15.31:0/4054288790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c160bb0 0x7f307c0b6170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 -- 172.21.15.31:0/4054288790 >> 172.21.15.31:0/4054288790 conn(0x7f307c09c600 msgr2=0x7f307c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 -- 172.21.15.31:0/4054288790 shutdown_connections 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.332+0000 7f308237d640 1 -- 172.21.15.31:0/4054288790 wait complete. 2024-01-19T04:56:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.333+0000 7f308237d640 1 Processor -- start 2024-01-19T04:56:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.334+0000 7f308237d640 1 -- start start 2024-01-19T04:56:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308237d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0cfd10 0x7f307c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308237d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c0d1600 0x7f307c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308237d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307c160bb0 0x7f307c15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308237d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c9de0 con 0x7f307c160bb0 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308237d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0cc3a0 con 0x7f307c0d1600 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308237d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0cb790 con 0x7f307c0cfd10 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307c160bb0 0x7f307c15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307c160bb0 0x7f307c15a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39874/0 (socket says 172.21.15.31:39874) 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 -- 172.21.15.31:0/2374218357 learned_addr learned my addr 172.21.15.31:0/2374218357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f307b7fe640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c0d1600 0x7f307c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f307bfff640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0cfd10 0x7f307c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 -- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0cfd10 msgr2=0x7f307c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0cfd10 0x7f307c0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 -- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c0d1600 msgr2=0x7f307c159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c0d1600 0x7f307c159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 -- 172.21.15.31:0/2374218357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009a70 con 0x7f307c160bb0 2024-01-19T04:56:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f307bfff640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0cfd10 0x7f307c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308091a640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307c160bb0 0x7f307c15a100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f307400efc0 tx=0x7f307400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f3078ff9640 1 -- 172.21.15.31:0/2374218357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074009cd0 con 0x7f307c160bb0 2024-01-19T04:56:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f3078ff9640 1 -- 172.21.15.31:0/2374218357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400f040 con 0x7f307c160bb0 2024-01-19T04:56:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3064009400 con 0x7f307c160bb0 2024-01-19T04:56:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.335+0000 7f3078ff9640 1 -- 172.21.15.31:0/2374218357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074003990 con 0x7f307c160bb0 2024-01-19T04:56:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.336+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c0cbd50 con 0x7f307c160bb0 2024-01-19T04:56:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.337+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c0cc550 con 0x7f307c160bb0 2024-01-19T04:56:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.337+0000 7f3078ff9640 1 -- 172.21.15.31:0/2374218357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3074014480 con 0x7f307c160bb0 2024-01-19T04:56:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.337+0000 7f3078ff9640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.338+0000 7f3078ff9640 1 -- 172.21.15.31:0/2374218357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f307408a390 con 0x7f307c160bb0 2024-01-19T04:56:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.338+0000 7f307bfff640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.338+0000 7f307bfff640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3064036000 tx=0x7f3064008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.342+0000 7f3078ff9640 1 -- 172.21.15.31:0/2374218357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f307c0cc550 con 0x7f307c160bb0 2024-01-19T04:56:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.521+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f307c0cc510 con 0x7f307c160bb0 2024-01-19T04:56:54.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.523+0000 7f3078ff9640 1 -- 172.21.15.31:0/2374218357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f30740550d0 con 0x7f307c160bb0 2024-01-19T04:56:54.522 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:54.523 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:54.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 msgr2=0x7f305c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3064036000 tx=0x7f3064008030 comp rx=0 tx=0).stop 2024-01-19T04:56:54.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307c160bb0 msgr2=0x7f307c15a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307c160bb0 0x7f307c15a100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f307400efc0 tx=0x7f307400c490 comp rx=0 tx=0).stop 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 shutdown_connections 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f305c06aed0 0x7f305c06d390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f307c160bb0 0x7f307c15a100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f307c0d1600 0x7f307c159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 --2- 172.21.15.31:0/2374218357 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f307c0cfd10 0x7f307c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 >> 172.21.15.31:0/2374218357 conn(0x7f307c09c600 msgr2=0x7f307c15f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 shutdown_connections 2024-01-19T04:56:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.525+0000 7f308237d640 1 -- 172.21.15.31:0/2374218357 wait complete. 2024-01-19T04:56:54.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:56:23.785927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:56:24.240600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-01-19T04:56:32.396495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:54.535 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.640+0000 7f74117f8640 1 Processor -- start 2024-01-19T04:56:54.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.640+0000 7f74117f8640 1 -- start start 2024-01-19T04:56:54.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f74117f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f740c15d700 0x7f740c0d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f74117f8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d11b0 0x7f740c0d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f74117f8640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0cef10 con 0x7f740c0d1e10 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f74117f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0d2240 con 0x7f740c15d700 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f74117f8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0ce300 con 0x7f740c0d11b0 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f740affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f740c15d700 0x7f740c0d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f740affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f740c15d700 0x7f740c0d3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39876/0 (socket says 172.21.15.31:39876) 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f740affd640 1 -- 172.21.15.31:0/4217086099 learned_addr learned my addr 172.21.15.31:0/4217086099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f7403fff640 1 --2- 172.21.15.31:0/4217086099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d11b0 0x7f740c0d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f740affd640 1 -- 172.21.15.31:0/4217086099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d11b0 msgr2=0x7f740c0d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f740affd640 1 --2- 172.21.15.31:0/4217086099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d11b0 0x7f740c0d18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f740affd640 1 -- 172.21.15.31:0/4217086099 >> v1:172.21.15.137:6789/0 conn(0x7f740c0d1e10 legacy=0x7f740c161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.641+0000 7f740affd640 1 -- 172.21.15.31:0/4217086099 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0ce5e0 con 0x7f740c15d700 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.642+0000 7f740affd640 1 --2- 172.21.15.31:0/4217086099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c0d3e10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009870 tx=0x7f73fc02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5d1c1a624d77ff server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.642+0000 7f740a7fc640 1 -- 172.21.15.31:0/4217086099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc02f840 con 0x7f740c15d700 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.642+0000 7f740a7fc640 1 -- 172.21.15.31:0/4217086099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc02f9a0 con 0x7f740c15d700 2024-01-19T04:56:54.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.642+0000 7f740a7fc640 1 -- 172.21.15.31:0/4217086099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc02fd30 con 0x7f740c15d700 2024-01-19T04:56:54.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.642+0000 7f74117f8640 1 -- 172.21.15.31:0/4217086099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 msgr2=0x7f740c0d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.642+0000 7f74117f8640 1 --2- 172.21.15.31:0/4217086099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c0d3e10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009870 tx=0x7f73fc02ba20 comp rx=0 tx=0).stop 2024-01-19T04:56:54.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.643+0000 7f74117f8640 1 -- 172.21.15.31:0/4217086099 shutdown_connections 2024-01-19T04:56:54.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.643+0000 7f74117f8640 1 --2- 172.21.15.31:0/4217086099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d11b0 0x7f740c0d18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.643+0000 7f74117f8640 1 --2- 172.21.15.31:0/4217086099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c0d3e10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.643+0000 7f74117f8640 1 -- 172.21.15.31:0/4217086099 >> 172.21.15.31:0/4217086099 conn(0x7f740c0d32f0 msgr2=0x7f740c0d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.643+0000 7f74117f8640 1 -- 172.21.15.31:0/4217086099 shutdown_connections 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.643+0000 7f74117f8640 1 -- 172.21.15.31:0/4217086099 wait complete. 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 Processor -- start 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 -- start start 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740c0d11b0 0x7f740c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d1e10 0x7f740c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c14f610 con 0x7f740c15d700 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0c73f0 con 0x7f740c0d11b0 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.644+0000 7f74117f8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0c9100 con 0x7f740c0d1e10 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39892/0 (socket says 172.21.15.31:39892) 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 -- 172.21.15.31:0/531356709 learned_addr learned my addr 172.21.15.31:0/531356709 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740affd640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740c0d11b0 0x7f740c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f7403fff640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d1e10 0x7f740c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 -- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d1e10 msgr2=0x7f740c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d1e10 0x7f740c0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 -- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740c0d11b0 msgr2=0x7f740c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740c0d11b0 0x7f740c14f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 -- 172.21.15.31:0/531356709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0094d0 con 0x7f740c15d700 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f740b7fe640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c158010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f740400c7d0 tx=0x7f740400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f74037fe640 1 -- 172.21.15.31:0/531356709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404003b70 con 0x7f740c15d700 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f74037fe640 1 -- 172.21.15.31:0/531356709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7404003cf0 con 0x7f740c15d700 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c0b46b0 con 0x7f740c15d700 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f74037fe640 1 -- 172.21.15.31:0/531356709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400aa90 con 0x7f740c15d700 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.645+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c0c6da0 con 0x7f740c15d700 2024-01-19T04:56:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.646+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c0c7580 con 0x7f740c15d700 2024-01-19T04:56:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.647+0000 7f74037fe640 1 -- 172.21.15.31:0/531356709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f740400abf0 con 0x7f740c15d700 2024-01-19T04:56:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.647+0000 7f74037fe640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f73ec06aed0 0x7f73ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.647+0000 7f74037fe640 1 -- 172.21.15.31:0/531356709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7404089470 con 0x7f740c15d700 2024-01-19T04:56:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.647+0000 7f740affd640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f73ec06aed0 0x7f73ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.648+0000 7f740affd640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f73ec06aed0 0x7f73ec06d390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f73fc036000 tx=0x7f73fc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.652+0000 7f74037fe640 1 -- 172.21.15.31:0/531356709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7404054260 con 0x7f740c15d700 2024-01-19T04:56:54.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.827+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f740c074900 con 0x7f740c15d700 2024-01-19T04:56:54.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.828+0000 7f74037fe640 1 -- 172.21.15.31:0/531356709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7404003e60 con 0x7f740c15d700 2024-01-19T04:56:54.827 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:54.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f73ec06aed0 msgr2=0x7f73ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f73ec06aed0 0x7f73ec06d390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f73fc036000 tx=0x7f73fc008030 comp rx=0 tx=0).stop 2024-01-19T04:56:54.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 msgr2=0x7f740c158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c158010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f740400c7d0 tx=0x7f740400cc90 comp rx=0 tx=0).stop 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 shutdown_connections 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f73ec06aed0 0x7f73ec06d390 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f740c15d700 0x7f740c158010 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f740c0d1e10 0x7f740c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.829+0000 7f74117f8640 1 --2- 172.21.15.31:0/531356709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740c0d11b0 0x7f740c14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.830+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 >> 172.21.15.31:0/531356709 conn(0x7f740c0d32f0 msgr2=0x7f740c15bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.830+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 shutdown_connections 2024-01-19T04:56:54.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.830+0000 7f74117f8640 1 -- 172.21.15.31:0/531356709 wait complete. 2024-01-19T04:56:54.840 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:54.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.944+0000 7f186779e640 1 Processor -- start 2024-01-19T04:56:54.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.945+0000 7f186779e640 1 -- start start 2024-01-19T04:56:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.945+0000 7f186779e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f186015d720 0x7f18600d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.945+0000 7f186779e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18600d3ea0 0x7f18600d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.945+0000 7f186779e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600cf160 con 0x7f18600d1a50 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.945+0000 7f186779e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600d1ec0 con 0x7f18600d3ea0 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.945+0000 7f186779e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600ce8c0 con 0x7f186015d720 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1864d39640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18600d3ea0 0x7f18600d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1865d3b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f18600d1a50 0x7f18600cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33964/0 (socket says 172.21.15.31:33964) 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1865d3b640 1 -- 172.21.15.31:0/2464032085 learned_addr learned my addr 172.21.15.31:0/2464032085 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f186553a640 1 --2- 172.21.15.31:0/2464032085 >> v2:172.21.15.137:3300/0 conn(0x7f186015d720 0x7f18600d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1864d39640 1 -- 172.21.15.31:0/2464032085 >> v2:172.21.15.137:3300/0 conn(0x7f186015d720 msgr2=0x7f18600d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1864d39640 1 --2- 172.21.15.31:0/2464032085 >> v2:172.21.15.137:3300/0 conn(0x7f186015d720 0x7f18600d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1864d39640 1 -- 172.21.15.31:0/2464032085 >> v1:172.21.15.31:6789/0 conn(0x7f18600d1a50 legacy=0x7f18600cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1857fff640 1 -- 172.21.15.31:0/2464032085 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568976377 0 0) 0x7f18600cf160 con 0x7f18600d1a50 2024-01-19T04:56:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.946+0000 7f1864d39640 1 -- 172.21.15.31:0/2464032085 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600ceba0 con 0x7f18600d3ea0 2024-01-19T04:56:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f1864d39640 1 --2- 172.21.15.31:0/2464032085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f18600d1510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f18600cf350 tx=0x7f185002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e986f3200f6f68ed server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f1857fff640 1 -- 172.21.15.31:0/2464032085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185002f050 con 0x7f18600d3ea0 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f1857fff640 1 -- 172.21.15.31:0/2464032085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850031700 con 0x7f18600d3ea0 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f1857fff640 1 -- 172.21.15.31:0/2464032085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850031a90 con 0x7f18600d3ea0 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f186779e640 1 -- 172.21.15.31:0/2464032085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 msgr2=0x7f18600d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f186779e640 1 --2- 172.21.15.31:0/2464032085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f18600d1510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f18600cf350 tx=0x7f185002b7a0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f186779e640 1 -- 172.21.15.31:0/2464032085 shutdown_connections 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f186779e640 1 --2- 172.21.15.31:0/2464032085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f18600d1510 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f186779e640 1 --2- 172.21.15.31:0/2464032085 >> v2:172.21.15.137:3300/0 conn(0x7f186015d720 0x7f18600d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f186779e640 1 -- 172.21.15.31:0/2464032085 >> 172.21.15.31:0/2464032085 conn(0x7f18600d3310 msgr2=0x7f18600d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.947+0000 7f186779e640 1 -- 172.21.15.31:0/2464032085 shutdown_connections 2024-01-19T04:56:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.948+0000 7f186779e640 1 -- 172.21.15.31:0/2464032085 wait complete. 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 Processor -- start 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 -- start start 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18600d1a50 0x7f18600cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f1860159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186015d720 0x7f18600cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600cbca0 con 0x7f18600d3ea0 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600ca3e0 con 0x7f186015d720 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f186779e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600ca6c0 con 0x7f18600d1a50 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f1864d39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f1860159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f1864d39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f1860159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39912/0 (socket says 172.21.15.31:39912) 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.949+0000 7f1864d39640 1 -- 172.21.15.31:0/469243507 learned_addr learned my addr 172.21.15.31:0/469243507 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f186553a640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18600d1a50 0x7f18600cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1865d3b640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186015d720 0x7f18600cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1864d39640 1 -- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18600d1a50 msgr2=0x7f18600cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1864d39640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18600d1a50 0x7f18600cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1864d39640 1 -- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186015d720 msgr2=0x7f18600cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1864d39640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186015d720 0x7f18600cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1864d39640 1 -- 172.21.15.31:0/469243507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1850009400 con 0x7f18600d3ea0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f186553a640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18600d1a50 0x7f18600cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1865d3b640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186015d720 0x7f18600cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1864d39640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f1860159650 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1850000c00 tx=0x7f18500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1855ffb640 1 -- 172.21.15.31:0/469243507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850033c30 con 0x7f18600d3ea0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1855ffb640 1 -- 172.21.15.31:0/469243507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850033db0 con 0x7f18600d3ea0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18600ca9a0 con 0x7f18600d3ea0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f1855ffb640 1 -- 172.21.15.31:0/469243507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185003c640 con 0x7f18600d3ea0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.950+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18600caf60 con 0x7f18600d3ea0 2024-01-19T04:56:54.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.951+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1860074900 con 0x7f18600d3ea0 2024-01-19T04:56:54.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.952+0000 7f1855ffb640 1 -- 172.21.15.31:0/469243507 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f185003a040 con 0x7f18600d3ea0 2024-01-19T04:56:54.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.952+0000 7f1855ffb640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f184406aed0 0x7f184406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:54.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.952+0000 7f1855ffb640 1 -- 172.21.15.31:0/469243507 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f18500ab0f0 con 0x7f18600d3ea0 2024-01-19T04:56:54.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.952+0000 7f186553a640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f184406aed0 0x7f184406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:54.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.953+0000 7f186553a640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f184406aed0 0x7f184406d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f185c006fd0 tx=0x7f185c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:54.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:54.956+0000 7f1855ffb640 1 -- 172.21.15.31:0/469243507 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1850075ec0 con 0x7f18600d3ea0 2024-01-19T04:56:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.132+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18600cffc0 con 0x7f18600d3ea0 2024-01-19T04:56:55.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.132+0000 7f1855ffb640 1 -- 172.21.15.31:0/469243507 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18500794e0 con 0x7f18600d3ea0 2024-01-19T04:56:55.132 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f184406aed0 msgr2=0x7f184406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f184406aed0 0x7f184406d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f185c006fd0 tx=0x7f185c008040 comp rx=0 tx=0).stop 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 msgr2=0x7f1860159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f1860159650 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1850000c00 tx=0x7f18500046c0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 shutdown_connections 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f184406aed0 0x7f184406d390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186015d720 0x7f18600cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18600d3ea0 0x7f1860159650 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 --2- 172.21.15.31:0/469243507 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f18600d1a50 0x7f18600cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 >> 172.21.15.31:0/469243507 conn(0x7f18600d3310 msgr2=0x7f186015c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:55.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 shutdown_connections 2024-01-19T04:56:55.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.134+0000 7f186779e640 1 -- 172.21.15.31:0/469243507 wait complete. 2024-01-19T04:56:55.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.249+0000 7fc901d97640 1 Processor -- start 2024-01-19T04:56:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.249+0000 7fc901d97640 1 -- start start 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc901d97640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8fc0ce090 0x7fc8fc0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc901d97640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc901d97640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8fc0cce50 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc901d97640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8fc0d03b0 con 0x7fc8fc0ce090 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc901d97640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc0cc240 con 0x7fc8fc0cd2e0 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8faffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8fc0ce090 0x7fc8fc0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8fb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc8fc15b6b0 0x7fc8fc0cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33976/0 (socket says 172.21.15.31:33976) 2024-01-19T04:56:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8fb7fe640 1 -- 172.21.15.31:0/3883422045 learned_addr learned my addr 172.21.15.31:0/3883422045 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:55.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8fbfff640 1 --2- 172.21.15.31:0/3883422045 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8faffd640 1 -- 172.21.15.31:0/3883422045 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0cd2e0 msgr2=0x7fc8fc0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:55.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8faffd640 1 --2- 172.21.15.31:0/3883422045 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8faffd640 1 -- 172.21.15.31:0/3883422045 >> v1:172.21.15.31:6789/0 conn(0x7fc8fc15b6b0 legacy=0x7fc8fc0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.250+0000 7fc8faffd640 1 -- 172.21.15.31:0/3883422045 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc0cc520 con 0x7fc8fc0ce090 2024-01-19T04:56:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.251+0000 7fc8faffd640 1 --2- 172.21.15.31:0/3883422045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc0ce090 0x7fc8fc0d1c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00097a0 tx=0x7fc8f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668641e456886950 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.251+0000 7fc8fa7fc640 1 -- 172.21.15.31:0/3883422045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0030610 con 0x7fc8fc0ce090 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.251+0000 7fc8fa7fc640 1 -- 172.21.15.31:0/3883422045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0030770 con 0x7fc8fc0ce090 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.251+0000 7fc8fa7fc640 1 -- 172.21.15.31:0/3883422045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0030b00 con 0x7fc8fc0ce090 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 -- 172.21.15.31:0/3883422045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc0ce090 msgr2=0x7fc8fc0d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 --2- 172.21.15.31:0/3883422045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc0ce090 0x7fc8fc0d1c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00097a0 tx=0x7fc8f002b960 comp rx=0 tx=0).stop 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 -- 172.21.15.31:0/3883422045 shutdown_connections 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 --2- 172.21.15.31:0/3883422045 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 --2- 172.21.15.31:0/3883422045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc0ce090 0x7fc8fc0d1c20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 -- 172.21.15.31:0/3883422045 >> 172.21.15.31:0/3883422045 conn(0x7fc8fc0d1100 msgr2=0x7fc8fc0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 -- 172.21.15.31:0/3883422045 shutdown_connections 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.252+0000 7fc901d97640 1 -- 172.21.15.31:0/3883422045 wait complete. 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.253+0000 7fc901d97640 1 Processor -- start 2024-01-19T04:56:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.253+0000 7fc901d97640 1 -- start start 2024-01-19T04:56:55.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc901d97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc901d97640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0ce090 0x7fc8fc154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc901d97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc15b6b0 0x7fc8fc0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc901d97640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1546c0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc901d97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc0c5280 con 0x7fc8fc0cd2e0 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc901d97640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc0c74f0 con 0x7fc8fc0ce090 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc15b6b0 0x7fc8fc0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc15b6b0 0x7fc8fc0c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39930/0 (socket says 172.21.15.31:39930) 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 -- 172.21.15.31:0/1045131437 learned_addr learned my addr 172.21.15.31:0/1045131437 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 -- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0ce090 msgr2=0x7fc8fc154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fb7fe640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0ce090 0x7fc8fc154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 -- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8fc0cd2e0 msgr2=0x7fc8fc0c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 -- 172.21.15.31:0/1045131437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f0009400 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.254+0000 7fc8fbfff640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc15b6b0 0x7fc8fc0c5760 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4007e40 tx=0x7fc8f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.255+0000 7fc8e3fff640 1 -- 172.21.15.31:0/1045131437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4014620 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.255+0000 7fc8e3fff640 1 -- 172.21.15.31:0/1045131437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f400ee90 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.255+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8fc0c77d0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.255+0000 7fc8e3fff640 1 -- 172.21.15.31:0/1045131437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40149e0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.255+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8fc0c5ca0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.256+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8fc0c7cf0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.256+0000 7fc8e3fff640 1 -- 172.21.15.31:0/1045131437 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc8f4014b40 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.257+0000 7fc8e3fff640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc8e806af00 0x7fc8e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.257+0000 7fc8e3fff640 1 -- 172.21.15.31:0/1045131437 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fc8f408a0e0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.257+0000 7fc8fb7fe640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc8e806af00 0x7fc8e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.257+0000 7fc8fb7fe640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc8e806af00 0x7fc8e806d3c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc1583d0 tx=0x7fc8ec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.261+0000 7fc8e3fff640 1 -- 172.21.15.31:0/1045131437 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc8fc0c7cf0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.437+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8fc0c7cf0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.437+0000 7fc8e3fff640 1 -- 172.21.15.31:0/1045131437 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8fc0c7cf0 con 0x7fc8fc15b6b0 2024-01-19T04:56:55.437 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc8e806af00 msgr2=0x7fc8e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc8e806af00 0x7fc8e806d3c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc1583d0 tx=0x7fc8ec009040 comp rx=0 tx=0).stop 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc15b6b0 msgr2=0x7fc8fc0c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc15b6b0 0x7fc8fc0c5760 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4007e40 tx=0x7fc8f400e4b0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 shutdown_connections 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc8e806af00 0x7fc8e806d3c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8fc15b6b0 0x7fc8fc0c5760 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8fc0ce090 0x7fc8fc154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 --2- 172.21.15.31:0/1045131437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8fc0cd2e0 0x7fc8fc0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 >> 172.21.15.31:0/1045131437 conn(0x7fc8fc0d1100 msgr2=0x7fc8fc15d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 shutdown_connections 2024-01-19T04:56:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.439+0000 7fc901d97640 1 -- 172.21.15.31:0/1045131437 wait complete. 2024-01-19T04:56:55.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-19T04:56:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.554+0000 7faed609b640 1 Processor -- start 2024-01-19T04:56:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.554+0000 7faed609b640 1 -- start start 2024-01-19T04:56:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faed609b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faed00d18e0 0x7faed015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faed609b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d3e50 0x7faed00d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faed609b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00d2200 con 0x7faed00d0ef0 2024-01-19T04:56:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faed609b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed015dee0 con 0x7faed00d18e0 2024-01-19T04:56:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faed609b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed015e1c0 con 0x7faed00d3e50 2024-01-19T04:56:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faecf7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faed00d18e0 0x7faed015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faecffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faed00d0ef0 0x7faed0161100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33980/0 (socket says 172.21.15.31:33980) 2024-01-19T04:56:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faecffff640 1 -- 172.21.15.31:0/2702731167 learned_addr learned my addr 172.21.15.31:0/2702731167 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faeceffd640 1 --2- 172.21.15.31:0/2702731167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d3e50 0x7faed00d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.555+0000 7faecf7fe640 1 -- 172.21.15.31:0/2702731167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d3e50 msgr2=0x7faed00d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.556+0000 7faecf7fe640 1 --2- 172.21.15.31:0/2702731167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d3e50 0x7faed00d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.556+0000 7faecf7fe640 1 -- 172.21.15.31:0/2702731167 >> v1:172.21.15.31:6789/0 conn(0x7faed00d0ef0 legacy=0x7faed0161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.556+0000 7faecf7fe640 1 -- 172.21.15.31:0/2702731167 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed015e4a0 con 0x7faed00d18e0 2024-01-19T04:56:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.556+0000 7faecf7fe640 1 --2- 172.21.15.31:0/2702731167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d18e0 0x7faed015d860 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faec00097a0 tx=0x7faec002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca619d45f3b5a851 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.556+0000 7faece7fc640 1 -- 172.21.15.31:0/2702731167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec002d040 con 0x7faed00d18e0 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.556+0000 7faece7fc640 1 -- 172.21.15.31:0/2702731167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec0031810 con 0x7faed00d18e0 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.556+0000 7faece7fc640 1 -- 172.21.15.31:0/2702731167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0031ba0 con 0x7faed00d18e0 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 -- 172.21.15.31:0/2702731167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d18e0 msgr2=0x7faed015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 --2- 172.21.15.31:0/2702731167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d18e0 0x7faed015d860 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faec00097a0 tx=0x7faec002b8b0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 -- 172.21.15.31:0/2702731167 shutdown_connections 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 --2- 172.21.15.31:0/2702731167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d3e50 0x7faed00d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 --2- 172.21.15.31:0/2702731167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d18e0 0x7faed015d860 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 -- 172.21.15.31:0/2702731167 >> 172.21.15.31:0/2702731167 conn(0x7faed009c600 msgr2=0x7faed00d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 -- 172.21.15.31:0/2702731167 shutdown_connections 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.557+0000 7faed609b640 1 -- 172.21.15.31:0/2702731167 wait complete. 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.558+0000 7faed609b640 1 Processor -- start 2024-01-19T04:56:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.558+0000 7faed609b640 1 -- start start 2024-01-19T04:56:55.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faed609b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d0ef0 0x7faed00b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faed609b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faed00d18e0 0x7faed0152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faed609b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d3e50 0x7faed00b8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faed609b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b6d40 con 0x7faed00d3e50 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faed609b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b60c0 con 0x7faed00d18e0 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faed609b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b63a0 con 0x7faed00d0ef0 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d3e50 0x7faed00b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d3e50 0x7faed00b8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39956/0 (socket says 172.21.15.31:39956) 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 -- 172.21.15.31:0/2631036564 learned_addr learned my addr 172.21.15.31:0/2631036564 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecf7fe640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d0ef0 0x7faed00b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 -- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d0ef0 msgr2=0x7faed00b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d0ef0 0x7faed00b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faeceffd640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faed00d18e0 0x7faed0152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 -- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faed00d18e0 msgr2=0x7faed0152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faed00d18e0 0x7faed0152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecffff640 1 -- 172.21.15.31:0/2631036564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0009400 con 0x7faed00d3e50 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faecf7fe640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d0ef0 0x7faed00b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.559+0000 7faeceffd640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faed00d18e0 0x7faed0152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.560+0000 7faecffff640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d3e50 0x7faed00b8090 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faec800caa0 tx=0x7faec800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.560+0000 7faeb7fff640 1 -- 172.21.15.31:0/2631036564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800ac00 con 0x7faed00d3e50 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.560+0000 7faeb7fff640 1 -- 172.21.15.31:0/2631036564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800ad60 con 0x7faed00d3e50 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.560+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed00b6680 con 0x7faed00d3e50 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.560+0000 7faeb7fff640 1 -- 172.21.15.31:0/2631036564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8012620 con 0x7faed00d3e50 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.560+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed00b76a0 con 0x7faed00d3e50 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.561+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed00b6e80 con 0x7faed00d3e50 2024-01-19T04:56:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.561+0000 7faeb7fff640 1 -- 172.21.15.31:0/2631036564 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7faec8007070 con 0x7faed00d3e50 2024-01-19T04:56:55.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.562+0000 7faeb7fff640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faebc06aed0 0x7faebc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.562+0000 7faeb7fff640 1 -- 172.21.15.31:0/2631036564 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7faec80896e0 con 0x7faed00d3e50 2024-01-19T04:56:55.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.562+0000 7faecf7fe640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faebc06aed0 0x7faebc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.562+0000 7faecf7fe640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faebc06aed0 0x7faebc06d390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faec0036000 tx=0x7faec003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.566+0000 7faeb7fff640 1 -- 172.21.15.31:0/2631036564 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7faed00b6e80 con 0x7faed00d3e50 2024-01-19T04:56:55.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.744+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7faed00b6e80 con 0x7faed00d3e50 2024-01-19T04:56:55.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.744+0000 7faeb7fff640 1 -- 172.21.15.31:0/2631036564 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7faec8017020 con 0x7faed00d3e50 2024-01-19T04:56:55.744 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faebc06aed0 msgr2=0x7faebc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faebc06aed0 0x7faebc06d390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faec0036000 tx=0x7faec003c040 comp rx=0 tx=0).stop 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d3e50 msgr2=0x7faed00b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d3e50 0x7faed00b8090 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faec800caa0 tx=0x7faec800cf60 comp rx=0 tx=0).stop 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 shutdown_connections 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faebc06aed0 0x7faebc06d390 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed00d3e50 0x7faed00b8090 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faed00d18e0 0x7faed0152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 --2- 172.21.15.31:0/2631036564 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faed00d0ef0 0x7faed00b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 >> 172.21.15.31:0/2631036564 conn(0x7faed009c600 msgr2=0x7faed00cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:55.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 shutdown_connections 2024-01-19T04:56:55.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.746+0000 7faed609b640 1 -- 172.21.15.31:0/2631036564 wait complete. 2024-01-19T04:56:55.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:56:55.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.859+0000 7f034a623640 1 Processor -- start 2024-01-19T04:56:55.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.859+0000 7f034a623640 1 -- start start 2024-01-19T04:56:55.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f034a623640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f034415d340 0x7f03440cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f034a623640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f03440cc490 0x7f034406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f034a623640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f034406aaf0 0x7f0344160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f034a623640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03440cc910 con 0x7f034406aaf0 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f034a623640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034406af80 con 0x7f03440cc490 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f034a623640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440c6130 con 0x7f034415d340 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f034406aaf0 0x7f0344160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f034406aaf0 0x7f0344160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39972/0 (socket says 172.21.15.31:39972) 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 -- 172.21.15.31:0/1761533749 learned_addr learned my addr 172.21.15.31:0/1761533749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f03437fe640 1 --2- 172.21.15.31:0/1761533749 >> v2:172.21.15.137:3300/0 conn(0x7f03440cc490 0x7f034406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0343fff640 1 --2- 172.21.15.31:0/1761533749 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f034415d340 0x7f03440cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 -- 172.21.15.31:0/1761533749 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f034415d340 msgr2=0x7f03440cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:55.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 --2- 172.21.15.31:0/1761533749 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f034415d340 0x7f03440cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 -- 172.21.15.31:0/1761533749 >> v2:172.21.15.137:3300/0 conn(0x7f03440cc490 msgr2=0x7f034406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:55.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 --2- 172.21.15.31:0/1761533749 >> v2:172.21.15.137:3300/0 conn(0x7f03440cc490 0x7f034406a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.860+0000 7f0348bc0640 1 -- 172.21.15.31:0/1761533749 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440ce630 con 0x7f034406aaf0 2024-01-19T04:56:55.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.861+0000 7f0348bc0640 1 --2- 172.21.15.31:0/1761533749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f0344160ff0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b3da4964b3ce5b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.861+0000 7f0342ffd640 1 -- 172.21.15.31:0/1761533749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c02c950 con 0x7f034406aaf0 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.861+0000 7f0342ffd640 1 -- 172.21.15.31:0/1761533749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c02cab0 con 0x7f034406aaf0 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.861+0000 7f0342ffd640 1 -- 172.21.15.31:0/1761533749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c02ce40 con 0x7f034406aaf0 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 -- 172.21.15.31:0/1761533749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 msgr2=0x7f0344160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 --2- 172.21.15.31:0/1761533749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f0344160ff0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c02bdb0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 -- 172.21.15.31:0/1761533749 shutdown_connections 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 --2- 172.21.15.31:0/1761533749 >> v2:172.21.15.137:3300/0 conn(0x7f03440cc490 0x7f034406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 --2- 172.21.15.31:0/1761533749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f0344160ff0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 --2- 172.21.15.31:0/1761533749 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f034415d340 0x7f03440cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 -- 172.21.15.31:0/1761533749 >> 172.21.15.31:0/1761533749 conn(0x7f03440b1c50 msgr2=0x7f03440b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 -- 172.21.15.31:0/1761533749 shutdown_connections 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.862+0000 7f034a623640 1 -- 172.21.15.31:0/1761533749 wait complete. 2024-01-19T04:56:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.863+0000 7f034a623640 1 Processor -- start 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.863+0000 7f034a623640 1 -- start start 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f03441f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f03440cc490 0x7f0344156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f034415d340 0x7f03440ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440c7530 con 0x7f034406aaf0 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440b4f20 con 0x7f034415d340 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440b4960 con 0x7f03440cc490 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f03441f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f03441f43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39976/0 (socket says 172.21.15.31:39976) 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 -- 172.21.15.31:0/890167552 learned_addr learned my addr 172.21.15.31:0/890167552 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f03437fe640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f03440cc490 0x7f0344156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0348bc0640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f034415d340 0x7f03440ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 -- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f03440cc490 msgr2=0x7f0344156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f03440cc490 0x7f0344156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 -- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f034415d340 msgr2=0x7f03440ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f034415d340 0x7f03440ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 -- 172.21.15.31:0/890167552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c009b30 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0343fff640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f03441f43d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f033400c7f0 tx=0x7f033400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0340ff9640 1 -- 172.21.15.31:0/890167552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334004220 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0340ff9640 1 -- 172.21.15.31:0/890167552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03340043a0 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f0340ff9640 1 -- 172.21.15.31:0/890167552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334007860 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03440b4c40 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.864+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0344156f90 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.865+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0344074900 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.866+0000 7f0340ff9640 1 -- 172.21.15.31:0/890167552 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f03340079c0 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.866+0000 7f0340ff9640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f033006aed0 0x7f033006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.866+0000 7f0340ff9640 1 -- 172.21.15.31:0/890167552 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0334089430 con 0x7f034406aaf0 2024-01-19T04:56:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.867+0000 7f03437fe640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f033006aed0 0x7f033006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:55.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.867+0000 7f03437fe640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f033006aed0 0x7f033006d390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f034406db70 tx=0x7f0338008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:55.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:55.870+0000 7f0340ff9640 1 -- 172.21.15.31:0/890167552 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0334054210 con 0x7f034406aaf0 2024-01-19T04:56:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.063+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f034406dba0 con 0x7f034406aaf0 2024-01-19T04:56:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.063+0000 7f0340ff9640 1 -- 172.21.15.31:0/890167552 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0334057830 con 0x7f034406aaf0 2024-01-19T04:56:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f033006aed0 msgr2=0x7f033006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f033006aed0 0x7f033006d390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f034406db70 tx=0x7f0338008040 comp rx=0 tx=0).stop 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 msgr2=0x7f03441f43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f03441f43d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f033400c7f0 tx=0x7f033400ccb0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 shutdown_connections 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f033006aed0 0x7f033006d390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f034415d340 0x7f03440ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f03440cc490 0x7f0344156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 --2- 172.21.15.31:0/890167552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034406aaf0 0x7f03441f43d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 >> 172.21.15.31:0/890167552 conn(0x7f03440b1c50 msgr2=0x7f03441573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.065+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 shutdown_connections 2024-01-19T04:56:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.066+0000 7f034a623640 1 -- 172.21.15.31:0/890167552 wait complete. 2024-01-19T04:56:56.075 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:56:56.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:56.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-19T04:56:56.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[53261]: fuse finished with error 0 and tester_r 0 2024-01-19T04:56:56.103 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:56:56.129 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:56:56.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:56.129 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:56:56.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-01-19T04:56:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.332+0000 7fd960f1a640 1 Processor -- start 2024-01-19T04:56:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.332+0000 7fd960f1a640 1 -- start start 2024-01-19T04:56:56.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd960f1a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 0x7fd95c15f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd960f1a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c06c640 con 0x7fd95c15edd0 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd960f1a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c06c920 con 0x7fd95c0cfa00 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd960f1a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c06cc00 con 0x7fd95c13dfc0 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd953fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd95c15edd0 0x7fd95c0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33992/0 (socket says 172.21.15.31:33992) 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd953fff640 1 -- 172.21.15.31:0/3127027152 learned_addr learned my addr 172.21.15.31:0/3127027152 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd95a59c640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 0x7fd95c15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463225073 0 0) 0x7fd95c06c640 con 0x7fd95c15edd0 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.333+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd948003520 con 0x7fd95c15edd0 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1683409865 0 0) 0x7fd948003520 con 0x7fd95c15edd0 2024-01-19T04:56:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 msgr2=0x7fd95c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 0x7fd95c15f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 >> v1:172.21.15.137:6789/0 conn(0x7fd95c0cfa00 legacy=0x7fd95c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c06cee0 con 0x7fd95c15edd0 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fd944001670 con 0x7fd95c15edd0 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 >> v1:172.21.15.31:6789/0 conn(0x7fd95c15edd0 legacy=0x7fd95c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9480039a0 0x7fd948003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 0x7fd9480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd948004d00 0x7fd948005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c06cc00 con 0x7fd948004d00 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800b980 con 0x7fd9480039a0 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800bb40 con 0x7fd95c13dfc0 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd95ad9d640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd948004d00 0x7fd948005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd95a59c640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9480039a0 0x7fd948003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd953fff640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 0x7fd9480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd95ad9d640 1 -- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 msgr2=0x7fd9480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd95ad9d640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c13dfc0 0x7fd9480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd95ad9d640 1 -- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9480039a0 msgr2=0x7fd948003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd95ad9d640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9480039a0 0x7fd948003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.334+0000 7fd95ad9d640 1 -- 172.21.15.31:0/3127027152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948003520 con 0x7fd948004d00 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.335+0000 7fd95ad9d640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd948004d00 0x7fd948005180 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0b1f10 tx=0x7fd9480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc8acdacc797c4a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.335+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95402d6b0 con 0x7fd948004d00 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.335+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95402d810 con 0x7fd948004d00 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.335+0000 7fd959d9b640 1 -- 172.21.15.31:0/3127027152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95402db80 con 0x7fd948004d00 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.336+0000 7fd960f1a640 1 -- 172.21.15.31:0/3127027152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd948004d00 msgr2=0x7fd948005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.336+0000 7fd960f1a640 1 --2- 172.21.15.31:0/3127027152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd948004d00 0x7fd948005180 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0b1f10 tx=0x7fd9480057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.336+0000 7fd95ad9d640 1 -- 172.21.15.31:0/3127027152 reap_dead start 2024-01-19T04:56:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.336+0000 7fd960f1a640 1 -- 172.21.15.31:0/3127027152 shutdown_connections 2024-01-19T04:56:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.336+0000 7fd960f1a640 1 -- 172.21.15.31:0/3127027152 >> 172.21.15.31:0/3127027152 conn(0x7fd95c06aa00 msgr2=0x7fd95c06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.336+0000 7fd960f1a640 1 -- 172.21.15.31:0/3127027152 shutdown_connections 2024-01-19T04:56:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.336+0000 7fd960f1a640 1 -- 172.21.15.31:0/3127027152 wait complete. 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.337+0000 7fd960f1a640 1 Processor -- start 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.337+0000 7fd960f1a640 1 -- start start 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd960f1a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c15a440 0x7fd95c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd960f1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd95c0b0be0 0x7fd95c13c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd960f1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c15a820 0x7fd95c13c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd960f1a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c13ceb0 con 0x7fd95c15a820 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd960f1a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c1f2b40 con 0x7fd95c0b0be0 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd960f1a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0b3520 con 0x7fd95c15a440 2024-01-19T04:56:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c15a820 0x7fd95c13c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c15a820 0x7fd95c13c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40004/0 (socket says 172.21.15.31:40004) 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 -- 172.21.15.31:0/881198208 learned_addr learned my addr 172.21.15.31:0/881198208 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd953fff640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd95c0b0be0 0x7fd95c13c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95a59c640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c15a440 0x7fd95c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 -- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c15a440 msgr2=0x7fd95c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c15a440 0x7fd95c0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 -- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd95c0b0be0 msgr2=0x7fd95c13c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd95c0b0be0 0x7fd95c13c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 -- 172.21.15.31:0/881198208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0b3800 con 0x7fd95c15a820 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd953fff640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd95c0b0be0 0x7fd95c13c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd95ad9d640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c15a820 0x7fd95c13c970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd954002010 tx=0x7fd9540015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd952ffd640 1 -- 172.21.15.31:0/881198208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954044070 con 0x7fd95c15a820 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd952ffd640 1 -- 172.21.15.31:0/881198208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954006e80 con 0x7fd95c15a820 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c0b3ae0 con 0x7fd95c15a820 2024-01-19T04:56:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.338+0000 7fd952ffd640 1 -- 172.21.15.31:0/881198208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95402ddb0 con 0x7fd95c15a820 2024-01-19T04:56:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.339+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c0b3180 con 0x7fd95c15a820 2024-01-19T04:56:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.340+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c074900 con 0x7fd95c15a820 2024-01-19T04:56:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.340+0000 7fd952ffd640 1 -- 172.21.15.31:0/881198208 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd954003970 con 0x7fd95c15a820 2024-01-19T04:56:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.340+0000 7fd952ffd640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd93c06af00 0x7fd93c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.340+0000 7fd952ffd640 1 -- 172.21.15.31:0/881198208 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd954031070 con 0x7fd95c15a820 2024-01-19T04:56:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.340+0000 7fd95a59c640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd93c06af00 0x7fd93c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.341+0000 7fd95a59c640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd93c06af00 0x7fd93c06d3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009b00 tx=0x7fd94c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:56.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.344+0000 7fd952ffd640 1 -- 172.21.15.31:0/881198208 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd954075b80 con 0x7fd95c15a820 2024-01-19T04:56:56.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.592+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd95c06e0a0 con 0x7fd95c15a820 2024-01-19T04:56:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.595+0000 7fd952ffd640 1 -- 172.21.15.31:0/881198208 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v12) v1 ==== 184+0+141 (secure 0 0 0) 0x7fd9540791a0 con 0x7fd95c15a820 2024-01-19T04:56:56.595 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:56:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:56:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2024-01-19T04:56:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:56:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw pool=cephfs_data" 2024-01-19T04:56:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:56:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd93c06af00 msgr2=0x7fd93c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd93c06af00 0x7fd93c06d3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009b00 tx=0x7fd94c008040 comp rx=0 tx=0).stop 2024-01-19T04:56:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c15a820 msgr2=0x7fd95c13c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c15a820 0x7fd95c13c970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd954002010 tx=0x7fd9540015b0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 shutdown_connections 2024-01-19T04:56:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd93c06af00 0x7fd93c06d3c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c15a820 0x7fd95c13c970 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd95c0b0be0 0x7fd95c13c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 --2- 172.21.15.31:0/881198208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd95c15a440 0x7fd95c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.597+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 >> 172.21.15.31:0/881198208 conn(0x7fd95c06aa00 msgr2=0x7fd95c15b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.598+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 shutdown_connections 2024-01-19T04:56:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:56.598+0000 7fd960f1a640 1 -- 172.21.15.31:0/881198208 wait complete. 2024-01-19T04:56:56.608 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-19T04:56:56.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:56:56.664 DEBUG:teuthology.orchestra.run.smithi137:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:56:56.709 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-19T04:56:56.709 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:56.709 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:56.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057151,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20760 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249472k,nr_inodes=812368,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-19T04:56:56.724 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:56:56.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:56.724 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:56:56.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:56:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-01-19T04:56:56.779 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:56:56.779 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:56.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:56:56.796 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:56:56.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:56.797 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-19T04:56:56.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:56:56.854 INFO:teuthology.orchestra.run.smithi137.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-01-19T04:56:56.854 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:56:56.854 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:56.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057151,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20760 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249472k,nr_inodes=812368,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-19T04:56:56.869 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:56:56.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:56.870 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-19T04:56:56.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:56:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-19T04:56:56.924 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:56:56.924 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:56:56.941 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:56:56.941 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:56:56.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:56:56.942 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-19T04:56:56.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:56:56.998 INFO:teuthology.orchestra.run.smithi137.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-19T04:56:56.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:57.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.099+0000 7f5ee8e75640 1 Processor -- start 2024-01-19T04:56:57.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.100+0000 7f5ee8e75640 1 -- start start 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.100+0000 7f5ee8e75640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5ee415d230 0x7f5ee40cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.100+0000 7f5ee8e75640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee406b3d0 con 0x7f5ee40c9520 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.100+0000 7f5ee8e75640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee4161400 con 0x7f5ee415d610 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.100+0000 7f5ee8e75640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee40cd300 con 0x7f5ee415d230 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee2d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5ee40c9520 0x7f5ee406acc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34002/0 (socket says 172.21.15.31:34002) 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee2d9d640 1 -- 172.21.15.31:0/3073312107 learned_addr learned my addr 172.21.15.31:0/3073312107 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603194195 0 0) 0x7f5ee406b3d0 con 0x7f5ee40c9520 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed0003520 con 0x7f5ee40c9520 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee259c640 1 --2- 172.21.15.31:0/3073312107 >> v2:172.21.15.137:3300/0 conn(0x7f5ee415d230 0x7f5ee40cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1137447571 0 0) 0x7f5ed0003520 con 0x7f5ee40c9520 2024-01-19T04:56:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 >> v2:172.21.15.137:3300/0 conn(0x7f5ee415d230 msgr2=0x7f5ee40cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 --2- 172.21.15.31:0/3073312107 >> v2:172.21.15.137:3300/0 conn(0x7f5ee415d230 0x7f5ee40cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 >> v1:172.21.15.137:6789/0 conn(0x7f5ee415d610 legacy=0x7f5ee406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee41616e0 con 0x7f5ee40c9520 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f5ed40019f0 con 0x7f5ee40c9520 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 >> v1:172.21.15.31:6789/0 conn(0x7f5ee40c9520 legacy=0x7f5ee406acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed00039c0 0x7f5ed0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00047c0 0x7f5ed0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed0005180 0x7f5ed000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40cd300 con 0x7f5ed0005180 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed000c080 con 0x7f5ed00047c0 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed000c240 con 0x7f5ed00039c0 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.101+0000 7f5ee2d9d640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed0005180 0x7f5ed000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee1d9b640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00047c0 0x7f5ed0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee259c640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed00039c0 0x7f5ed0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee2d9d640 1 -- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed00039c0 msgr2=0x7f5ed0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee2d9d640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed00039c0 0x7f5ed0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee2d9d640 1 -- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00047c0 msgr2=0x7f5ed0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee2d9d640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00047c0 0x7f5ed0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee2d9d640 1 -- 172.21.15.31:0/3073312107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed0003520 con 0x7f5ed0005180 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee2d9d640 1 -- 172.21.15.31:0/3073312107 reap_dead start 2024-01-19T04:56:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.102+0000 7f5ee2d9d640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed0005180 0x7f5ed000b980 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400a760 tx=0x7f5ed00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d248ce520b75cd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4034630 con 0x7f5ed0005180 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed402f070 con 0x7f5ed0005180 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee159a640 1 -- 172.21.15.31:0/3073312107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4034a10 con 0x7f5ed0005180 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee8e75640 1 -- 172.21.15.31:0/3073312107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed0005180 msgr2=0x7f5ed000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed0005180 0x7f5ed000b980 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400a760 tx=0x7f5ed00057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee8e75640 1 -- 172.21.15.31:0/3073312107 shutdown_connections 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/3073312107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed0005180 0x7f5ed000b980 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee8e75640 1 -- 172.21.15.31:0/3073312107 >> 172.21.15.31:0/3073312107 conn(0x7f5ee40cc1b0 msgr2=0x7f5ee40cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee8e75640 1 -- 172.21.15.31:0/3073312107 shutdown_connections 2024-01-19T04:56:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.103+0000 7f5ee8e75640 1 -- 172.21.15.31:0/3073312107 wait complete. 2024-01-19T04:56:57.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 Processor -- start 2024-01-19T04:56:57.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 -- start start 2024-01-19T04:56:57.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed0005180 0x7f5ee40ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee4154740 0x7f5ee4151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ee4154bc0 0x7f5ee4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4152ce0 con 0x7f5ee4154740 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41520d0 con 0x7f5ee4154bc0 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee8e75640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41523b0 con 0x7f5ed0005180 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee4154740 0x7f5ee4151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee4154740 0x7f5ee4151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40026/0 (socket says 172.21.15.31:40026) 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.105+0000 7f5ee1d9b640 1 -- 172.21.15.31:0/653738677 learned_addr learned my addr 172.21.15.31:0/653738677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee259c640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed0005180 0x7f5ee40ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee2d9d640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ee4154bc0 0x7f5ee4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee1d9b640 1 -- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed0005180 msgr2=0x7f5ee40ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee1d9b640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed0005180 0x7f5ee40ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee1d9b640 1 -- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ee4154bc0 msgr2=0x7f5ee4151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee1d9b640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ee4154bc0 0x7f5ee4151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee1d9b640 1 -- 172.21.15.31:0/653738677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4152690 con 0x7f5ee4154740 2024-01-19T04:56:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee2d9d640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ee4154bc0 0x7f5ee4151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee1d9b640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee4154740 0x7f5ee4151100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc007810 tx=0x7f5ecc00f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5edaffd640 1 -- 172.21.15.31:0/653738677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc0052e0 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5edaffd640 1 -- 172.21.15.31:0/653738677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc005440 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5edaffd640 1 -- 172.21.15.31:0/653738677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc003830 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee4152970 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.106+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee40ce7a0 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.107+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee40ce9c0 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.108+0000 7f5edaffd640 1 -- 172.21.15.31:0/653738677 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5ecc003990 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.108+0000 7f5edaffd640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5ec406af00 0x7f5ec406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.108+0000 7f5edaffd640 1 -- 172.21.15.31:0/653738677 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f5ecc089540 con 0x7f5ee4154740 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.108+0000 7f5ee259c640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5ec406af00 0x7f5ec406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.109+0000 7f5ee259c640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5ec406af00 0x7f5ec406d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009b40 tx=0x7f5edc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.112+0000 7f5edaffd640 1 -- 172.21.15.31:0/653738677 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5ecc054280 con 0x7f5ee4154740 2024-01-19T04:56:57.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.333+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ee40696e0 con 0x7f5ee4154740 2024-01-19T04:56:57.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.334+0000 7f5edaffd640 1 -- 172.21.15.31:0/653738677 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f5ecc0578a0 con 0x7f5ee4154740 2024-01-19T04:56:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:57.333 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:56:25.278275+0000","modified":"2024-01-19T04:56:32.866830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5259":{"gid":5259,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/968278604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":968278604},{"type":"v1","addr":"172.21.15.31:6835","nonce":968278604}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-19T04:56:57.334 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5ec406af00 msgr2=0x7f5ec406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5ec406af00 0x7f5ec406d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009b40 tx=0x7f5edc009190 comp rx=0 tx=0).stop 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee4154740 msgr2=0x7f5ee4151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee4154740 0x7f5ee4151100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc007810 tx=0x7f5ecc00f990 comp rx=0 tx=0).stop 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 shutdown_connections 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5ec406af00 0x7f5ec406d3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ee4154bc0 0x7f5ee4151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee4154740 0x7f5ee4151100 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 --2- 172.21.15.31:0/653738677 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ed0005180 0x7f5ee40ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 >> 172.21.15.31:0/653738677 conn(0x7f5ee40cc1b0 msgr2=0x7f5ee40c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 shutdown_connections 2024-01-19T04:56:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.336+0000 7f5ee8e75640 1 -- 172.21.15.31:0/653738677 wait complete. 2024-01-19T04:56:57.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:57.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.448+0000 7f49f333c640 1 Processor -- start 2024-01-19T04:56:57.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.448+0000 7f49f333c640 1 -- start start 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f333c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49ec160bb0 0x7f49ec0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f333c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec0b66b0 con 0x7f49ec0cfd10 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f333c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec0d1b90 con 0x7f49ec0d1600 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f333c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ec0d1e70 con 0x7f49ec160bb0 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f08d7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f49ec0cfd10 0x7f49ec0d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34004/0 (socket says 172.21.15.31:34004) 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f08d7640 1 -- 172.21.15.31:0/2730512972 learned_addr learned my addr 172.21.15.31:0/2730512972 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f10d8640 1 --2- 172.21.15.31:0/2730512972 >> v2:172.21.15.137:3300/0 conn(0x7f49ec160bb0 0x7f49ec0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49e3fff640 1 -- 172.21.15.31:0/2730512972 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056445447 0 0) 0x7f49ec0b66b0 con 0x7f49ec0cfd10 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49e3fff640 1 -- 172.21.15.31:0/2730512972 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d8003520 con 0x7f49ec0cfd10 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49e3fff640 1 -- 172.21.15.31:0/2730512972 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3239516812 0 0) 0x7f49d8003520 con 0x7f49ec0cfd10 2024-01-19T04:56:57.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f10d8640 1 -- 172.21.15.31:0/2730512972 >> v1:172.21.15.137:6789/0 conn(0x7f49ec0d1600 legacy=0x7f49ec0ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f10d8640 1 -- 172.21.15.31:0/2730512972 >> v1:172.21.15.31:6789/0 conn(0x7f49ec0cfd10 legacy=0x7f49ec0d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.449+0000 7f49f10d8640 1 -- 172.21.15.31:0/2730512972 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec0d2150 con 0x7f49ec160bb0 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.450+0000 7f49f10d8640 1 --2- 172.21.15.31:0/2730512972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0b6170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49e80097a0 tx=0x7f49e802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d7b03487862dd1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.450+0000 7f49e3fff640 1 -- 172.21.15.31:0/2730512972 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e802f050 con 0x7f49ec160bb0 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.450+0000 7f49e3fff640 1 -- 172.21.15.31:0/2730512972 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e8031500 con 0x7f49ec160bb0 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.450+0000 7f49e3fff640 1 -- 172.21.15.31:0/2730512972 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc001670 con 0x7f49ec160bb0 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.451+0000 7f49f333c640 1 -- 172.21.15.31:0/2730512972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 msgr2=0x7f49ec0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.451+0000 7f49f333c640 1 --2- 172.21.15.31:0/2730512972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0b6170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49e80097a0 tx=0x7f49e802b820 comp rx=0 tx=0).stop 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.451+0000 7f49f333c640 1 -- 172.21.15.31:0/2730512972 shutdown_connections 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.451+0000 7f49f333c640 1 --2- 172.21.15.31:0/2730512972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0b6170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.451+0000 7f49f333c640 1 -- 172.21.15.31:0/2730512972 >> 172.21.15.31:0/2730512972 conn(0x7f49ec09c600 msgr2=0x7f49ec0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:57.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.451+0000 7f49f333c640 1 -- 172.21.15.31:0/2730512972 shutdown_connections 2024-01-19T04:56:57.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.451+0000 7f49f333c640 1 -- 172.21.15.31:0/2730512972 wait complete. 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.452+0000 7f49f333c640 1 Processor -- start 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.452+0000 7f49f333c640 1 -- start start 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f333c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49ec0cfd10 0x7f49ec153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f333c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49ec0d1600 0x7f49ec0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f333c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f333c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c7e40 con 0x7f49ec0d1600 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f333c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c72c0 con 0x7f49ec160bb0 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f333c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c75a0 con 0x7f49ec0cfd10 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49ec0d1600 0x7f49ec0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49ec0d1600 0x7f49ec0cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40040/0 (socket says 172.21.15.31:40040) 2024-01-19T04:56:57.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 -- 172.21.15.31:0/3600343279 learned_addr learned my addr 172.21.15.31:0/3600343279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f18d9640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f10d8640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49ec0cfd10 0x7f49ec153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 -- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49ec0cfd10 msgr2=0x7f49ec153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49ec0cfd10 0x7f49ec153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 -- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 msgr2=0x7f49ec0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 -- 172.21.15.31:0/3600343279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e8009400 con 0x7f49ec0d1600 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f18d9640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.453+0000 7f49f08d7640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49ec0d1600 0x7f49ec0cc5c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0011e0 tx=0x7f49dc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.454+0000 7f49e1ffb640 1 -- 172.21.15.31:0/3600343279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc005ad0 con 0x7f49ec0d1600 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.454+0000 7f49e1ffb640 1 -- 172.21.15.31:0/3600343279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc005c50 con 0x7f49ec0d1600 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.454+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec0c7880 con 0x7f49ec0d1600 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.454+0000 7f49e1ffb640 1 -- 172.21.15.31:0/3600343279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0072b0 con 0x7f49ec0d1600 2024-01-19T04:56:57.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.454+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec0c6e50 con 0x7f49ec0d1600 2024-01-19T04:56:57.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.455+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec0c7070 con 0x7f49ec0d1600 2024-01-19T04:56:57.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.455+0000 7f49e1ffb640 1 -- 172.21.15.31:0/3600343279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f49dc007410 con 0x7f49ec0d1600 2024-01-19T04:56:57.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.455+0000 7f49e1ffb640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f49d006aed0 0x7f49d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.455+0000 7f49e1ffb640 1 -- 172.21.15.31:0/3600343279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f49dc088dc0 con 0x7f49ec0d1600 2024-01-19T04:56:57.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.456+0000 7f49f10d8640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f49d006aed0 0x7f49d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.456+0000 7f49f10d8640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f49d006aed0 0x7f49d006d390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f49e8035000 tx=0x7f49e8008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.460+0000 7f49e1ffb640 1 -- 172.21.15.31:0/3600343279 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f49dc053b90 con 0x7f49ec0d1600 2024-01-19T04:56:57.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.642+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49ec0696e0 con 0x7f49ec0d1600 2024-01-19T04:56:57.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.643+0000 7f49e1ffb640 1 -- 172.21.15.31:0/3600343279 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f49dc017020 con 0x7f49ec0d1600 2024-01-19T04:56:57.643 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:57.643 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f49d006aed0 msgr2=0x7f49d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f49d006aed0 0x7f49d006d390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f49e8035000 tx=0x7f49e8008030 comp rx=0 tx=0).stop 2024-01-19T04:56:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49ec0d1600 msgr2=0x7f49ec0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49ec0d1600 0x7f49ec0cc5c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0011e0 tx=0x7f49dc00cb90 comp rx=0 tx=0).stop 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 shutdown_connections 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f49d006aed0 0x7f49d006d390 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec160bb0 0x7f49ec0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49ec0d1600 0x7f49ec0cc5c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 --2- 172.21.15.31:0/3600343279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49ec0cfd10 0x7f49ec153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 >> 172.21.15.31:0/3600343279 conn(0x7f49ec09c600 msgr2=0x7f49ec0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 shutdown_connections 2024-01-19T04:56:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.646+0000 7f49f333c640 1 -- 172.21.15.31:0/3600343279 wait complete. 2024-01-19T04:56:57.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:56:23.785927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:56:24.240600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-01-19T04:56:32.396495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:56:57.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:56:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.760+0000 7f8e36794640 1 Processor -- start 2024-01-19T04:56:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.760+0000 7f8e36794640 1 -- start start 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e36794640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8e300d3e50 0x7f8e3015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e36794640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3015dee0 con 0x7f8e300d0ef0 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e36794640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3015e1c0 con 0x7f8e300d18e0 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e36794640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3015e4a0 con 0x7f8e300d3e50 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e34d31640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8e300d0ef0 0x7f8e300cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34016/0 (socket says 172.21.15.31:34016) 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e34d31640 1 -- 172.21.15.31:0/2094883348 learned_addr learned my addr 172.21.15.31:0/2094883348 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526342250 0 0) 0x7f8e3015dee0 con 0x7f8e300d0ef0 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2f7fe640 1 --2- 172.21.15.31:0/2094883348 >> v2:172.21.15.137:3300/0 conn(0x7f8e300d3e50 0x7f8e3015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e1c003520 con 0x7f8e300d0ef0 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2973682960 0 0) 0x7f8e1c003520 con 0x7f8e300d0ef0 2024-01-19T04:56:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 >> v2:172.21.15.137:3300/0 conn(0x7f8e300d3e50 msgr2=0x7f8e3015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 --2- 172.21.15.31:0/2094883348 >> v2:172.21.15.137:3300/0 conn(0x7f8e300d3e50 0x7f8e3015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 >> v1:172.21.15.137:6789/0 conn(0x7f8e300d18e0 legacy=0x7f8e300d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3015e780 con 0x7f8e300d0ef0 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f8e280019f0 con 0x7f8e300d0ef0 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 >> v1:172.21.15.31:6789/0 conn(0x7f8e300d0ef0 legacy=0x7f8e300cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e1c0047c0 0x7f8e1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e1c005180 0x7f8e1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3015e4a0 con 0x7f8e1c0039c0 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c00c080 con 0x7f8e1c0047c0 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.761+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c00c240 con 0x7f8e1c005180 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e2ffff640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e2f7fe640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e1c0047c0 0x7f8e1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e34d31640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e1c005180 0x7f8e1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e2ffff640 1 -- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e1c005180 msgr2=0x7f8e1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e2ffff640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e1c005180 0x7f8e1c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e2ffff640 1 -- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e1c0047c0 msgr2=0x7f8e1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e2ffff640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e1c0047c0 0x7f8e1c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e2ffff640 1 -- 172.21.15.31:0/2094883348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c003520 con 0x7f8e1c0039c0 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.762+0000 7f8e34d31640 1 -- 172.21.15.31:0/2094883348 reap_dead start 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e2ffff640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e1c003de0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8e18001750 tx=0x7f8e1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f31ff917a558c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180052d0 con 0x7f8e1c0039c0 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1802e050 con 0x7f8e1c0039c0 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e2effd640 1 -- 172.21.15.31:0/2094883348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18003830 con 0x7f8e1c0039c0 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e36794640 1 -- 172.21.15.31:0/2094883348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 msgr2=0x7f8e1c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e36794640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e1c003de0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8e18001750 tx=0x7f8e1c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e36794640 1 -- 172.21.15.31:0/2094883348 shutdown_connections 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e36794640 1 --2- 172.21.15.31:0/2094883348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e1c003de0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e36794640 1 -- 172.21.15.31:0/2094883348 >> 172.21.15.31:0/2094883348 conn(0x7f8e3009c600 msgr2=0x7f8e300d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:57.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e36794640 1 -- 172.21.15.31:0/2094883348 shutdown_connections 2024-01-19T04:56:57.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.763+0000 7f8e36794640 1 -- 172.21.15.31:0/2094883348 wait complete. 2024-01-19T04:56:57.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.764+0000 7f8e36794640 1 Processor -- start 2024-01-19T04:56:57.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e36794640 1 -- start start 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e36794640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e300c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e36794640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e300cbe10 0x7f8e30154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e36794640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e300b01b0 0x7f8e300b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e36794640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30154950 con 0x7f8e1c0039c0 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e36794640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300c7550 con 0x7f8e300cbe10 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e36794640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300c7830 con 0x7f8e300b01b0 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.765+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e300c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e300c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40064/0 (socket says 172.21.15.31:40064) 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 -- 172.21.15.31:0/1749965901 learned_addr learned my addr 172.21.15.31:0/1749965901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2f7fe640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e300cbe10 0x7f8e30154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e34d31640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e300b01b0 0x7f8e300b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 -- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e300b01b0 msgr2=0x7f8e300b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e300b01b0 0x7f8e300b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 -- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e300cbe10 msgr2=0x7f8e30154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e300cbe10 0x7f8e30154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 -- 172.21.15.31:0/1749965901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300c7b10 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2ffff640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e300c6e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8e18000c00 tx=0x7f8e18001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2cff9640 1 -- 172.21.15.31:0/1749965901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180076c0 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2cff9640 1 -- 172.21.15.31:0/1749965901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e18007a30 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e300c7df0 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e2cff9640 1 -- 172.21.15.31:0/1749965901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18006c00 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.766+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e300cc290 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.767+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e30074900 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.768+0000 7f8e2cff9640 1 -- 172.21.15.31:0/1749965901 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8e180035f0 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.768+0000 7f8e2cff9640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006af00 0x7f8e1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.768+0000 7f8e2cff9640 1 -- 172.21.15.31:0/1749965901 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f8e180ab280 con 0x7f8e1c0039c0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.768+0000 7f8e2f7fe640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006af00 0x7f8e1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:57.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.769+0000 7f8e2f7fe640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006af00 0x7f8e1006d3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8e30159900 tx=0x7f8e20009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:57.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.772+0000 7f8e2cff9640 1 -- 172.21.15.31:0/1749965901 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8e18075f30 con 0x7f8e1c0039c0 2024-01-19T04:56:57.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.949+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e300d3c30 con 0x7f8e1c0039c0 2024-01-19T04:56:57.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.950+0000 7f8e2cff9640 1 -- 172.21.15.31:0/1749965901 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e18079550 con 0x7f8e1c0039c0 2024-01-19T04:56:57.950 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:56:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.951+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006af00 msgr2=0x7f8e1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.951+0000 7f8e36794640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006af00 0x7f8e1006d3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8e30159900 tx=0x7f8e20009190 comp rx=0 tx=0).stop 2024-01-19T04:56:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.951+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 msgr2=0x7f8e300c6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.951+0000 7f8e36794640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e300c6e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8e18000c00 tx=0x7f8e18001120 comp rx=0 tx=0).stop 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 shutdown_connections 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8e1006af00 0x7f8e1006d3c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8e300b01b0 0x7f8e300b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e300cbe10 0x7f8e30154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 --2- 172.21.15.31:0/1749965901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1c0039c0 0x7f8e300c6e50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 >> 172.21.15.31:0/1749965901 conn(0x7f8e3009c600 msgr2=0x7f8e30158830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 shutdown_connections 2024-01-19T04:56:57.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:57.952+0000 7f8e36794640 1 -- 172.21.15.31:0/1749965901 wait complete. 2024-01-19T04:56:57.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:56:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.066+0000 7f618f73b640 1 Processor -- start 2024-01-19T04:56:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.067+0000 7f618f73b640 1 -- start start 2024-01-19T04:56:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.067+0000 7f618f73b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f618815d3a0 0x7f618806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.067+0000 7f618f73b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f61880cf570 0x7f6188161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.067+0000 7f618f73b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880cf9f0 con 0x7f618815d780 2024-01-19T04:56:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.067+0000 7f618f73b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61880cfcd0 con 0x7f61880cf570 2024-01-19T04:56:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.067+0000 7f618f73b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880cffb0 con 0x7f618815d3a0 2024-01-19T04:56:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618d4d7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f618815d3a0 0x7f618806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618ccd6640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f61880cf570 0x7f6188161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618dcd8640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f618815d780 0x7f61880cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:58150/0 (socket says 172.21.15.31:58150) 2024-01-19T04:56:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618dcd8640 1 -- 172.21.15.31:0/813065759 learned_addr learned my addr 172.21.15.31:0/813065759 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618d4d7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f618815d3a0 0x7f618806c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:35736/0 (socket says 172.21.15.31:35736) 2024-01-19T04:56:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618ccd6640 1 -- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f618815d3a0 msgr2=0x7f618806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618ccd6640 1 --2- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f618815d3a0 0x7f618806c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618ccd6640 1 -- 172.21.15.31:0/813065759 >> v1:172.21.15.137:6789/0 conn(0x7f618815d780 legacy=0x7f61880cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618ccd6640 1 -- 172.21.15.31:0/813065759 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880d0290 con 0x7f61880cf570 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.068+0000 7f618d4d7640 1 --2- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f618815d3a0 0x7f618806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.069+0000 7f618ccd6640 1 --2- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61880cf570 0x7f6188161150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f61780097a0 tx=0x7f617802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c612f3cef4543fd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.069+0000 7f617ffff640 1 -- 172.21.15.31:0/813065759 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617802d040 con 0x7f61880cf570 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.069+0000 7f617ffff640 1 -- 172.21.15.31:0/813065759 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178031810 con 0x7f61880cf570 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.069+0000 7f617ffff640 1 -- 172.21.15.31:0/813065759 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178031ba0 con 0x7f61880cf570 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 -- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61880cf570 msgr2=0x7f6188161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 --2- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61880cf570 0x7f6188161150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f61780097a0 tx=0x7f617802b8b0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 -- 172.21.15.31:0/813065759 shutdown_connections 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 --2- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61880cf570 0x7f6188161150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 --2- 172.21.15.31:0/813065759 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f618815d3a0 0x7f618806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 -- 172.21.15.31:0/813065759 >> 172.21.15.31:0/813065759 conn(0x7f61880d1100 msgr2=0x7f61880d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 -- 172.21.15.31:0/813065759 shutdown_connections 2024-01-19T04:56:58.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.070+0000 7f618f73b640 1 -- 172.21.15.31:0/813065759 wait complete. 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.071+0000 7f618f73b640 1 Processor -- start 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.071+0000 7f618f73b640 1 -- start start 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618f73b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61880cf570 0x7f61880b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618f73b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618815d3a0 0x7f61881493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618f73b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f618815d780 0x7f6188156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618f73b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6188152810 con 0x7f618815d780 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618f73b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880b1cc0 con 0x7f618815d3a0 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618f73b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880b1fa0 con 0x7f61880cf570 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f618815d780 0x7f6188156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f618815d780 0x7f6188156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40074/0 (socket says 172.21.15.31:40074) 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 -- 172.21.15.31:0/609313259 learned_addr learned my addr 172.21.15.31:0/609313259 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618ccd6640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618815d3a0 0x7f61881493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618d4d7640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61880cf570 0x7f61880b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 -- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61880cf570 msgr2=0x7f61880b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61880cf570 0x7f61880b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 -- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618815d3a0 msgr2=0x7f61881493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618815d3a0 0x7f61881493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 -- 172.21.15.31:0/609313259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184009a70 con 0x7f618815d780 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618d4d7640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61880cf570 0x7f61880b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618ccd6640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618815d3a0 0x7f61881493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.072+0000 7f618dcd8640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f618815d780 0x7f6188156100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f618000c370 tx=0x7f618000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.073+0000 7f617dffb640 1 -- 172.21.15.31:0/609313259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180005ce0 con 0x7f618815d780 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.073+0000 7f617dffb640 1 -- 172.21.15.31:0/609313259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180005e40 con 0x7f618815d780 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.073+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6178009400 con 0x7f618815d780 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.073+0000 7f617dffb640 1 -- 172.21.15.31:0/609313259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61800072d0 con 0x7f618815d780 2024-01-19T04:56:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.073+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61880b2560 con 0x7f618815d780 2024-01-19T04:56:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.074+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6188074900 con 0x7f618815d780 2024-01-19T04:56:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.074+0000 7f617dffb640 1 -- 172.21.15.31:0/609313259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f61800074f0 con 0x7f618815d780 2024-01-19T04:56:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.075+0000 7f617dffb640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f616c06aed0 0x7f616c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.075+0000 7f617dffb640 1 -- 172.21.15.31:0/609313259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6180088bd0 con 0x7f618815d780 2024-01-19T04:56:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.075+0000 7f618d4d7640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f616c06aed0 0x7f616c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.075+0000 7f618d4d7640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f616c06aed0 0x7f616c06d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6184009fd0 tx=0x7f618400b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.079+0000 7f617dffb640 1 -- 172.21.15.31:0/609313259 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f61800539c0 con 0x7f618815d780 2024-01-19T04:56:58.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.255+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6188067030 con 0x7f618815d780 2024-01-19T04:56:58.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.256+0000 7f617dffb640 1 -- 172.21.15.31:0/609313259 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6180017020 con 0x7f618815d780 2024-01-19T04:56:58.256 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:58.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.257+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f616c06aed0 msgr2=0x7f616c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.257+0000 7f618f73b640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f616c06aed0 0x7f616c06d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6184009fd0 tx=0x7f618400b040 comp rx=0 tx=0).stop 2024-01-19T04:56:58.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f618815d780 msgr2=0x7f6188156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f618815d780 0x7f6188156100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f618000c370 tx=0x7f618000c830 comp rx=0 tx=0).stop 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 shutdown_connections 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f616c06aed0 0x7f616c06d390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f618815d780 0x7f6188156100 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618815d3a0 0x7f61881493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 --2- 172.21.15.31:0/609313259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f61880cf570 0x7f61880b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 >> 172.21.15.31:0/609313259 conn(0x7f61880d1100 msgr2=0x7f618815f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 shutdown_connections 2024-01-19T04:56:58.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.258+0000 7f618f73b640 1 -- 172.21.15.31:0/609313259 wait complete. 2024-01-19T04:56:58.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:56:58.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.373+0000 7f9ea3c7c640 1 Processor -- start 2024-01-19T04:56:58.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.373+0000 7f9ea3c7c640 1 -- start start 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea3c7c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9e9c161860 0x7f9e9c168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea3c7c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c0cf9f0 con 0x7f9e9c15d3a0 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea3c7c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c0cfcd0 con 0x7f9e9c0cf570 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea3c7c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9c0cffb0 con 0x7f9e9c161860 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea1a18640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9e9c15d3a0 0x7f9e9c06c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34024/0 (socket says 172.21.15.31:34024) 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea1a18640 1 -- 172.21.15.31:0/1785857830 learned_addr learned my addr 172.21.15.31:0/1785857830 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea2219640 1 --2- 172.21.15.31:0/1785857830 >> v2:172.21.15.137:3300/0 conn(0x7f9e9c161860 0x7f9e9c168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227392990 0 0) 0x7f9e9c0cf9f0 con 0x7f9e9c15d3a0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e88003520 con 0x7f9e9c15d3a0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240903013 0 0) 0x7f9e88003520 con 0x7f9e9c15d3a0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 >> v2:172.21.15.137:3300/0 conn(0x7f9e9c161860 msgr2=0x7f9e9c168490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 --2- 172.21.15.31:0/1785857830 >> v2:172.21.15.137:3300/0 conn(0x7f9e9c161860 0x7f9e9c168490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 >> v1:172.21.15.137:6789/0 conn(0x7f9e9c0cf570 legacy=0x7f9e9c161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0d0290 con 0x7f9e9c15d3a0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f9e98001b70 con 0x7f9e9c15d3a0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 >> v1:172.21.15.31:6789/0 conn(0x7f9e9c15d3a0 legacy=0x7f9e9c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880039c0 0x7f9e88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e880047c0 0x7f9e88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e880051d0 0x7f9e8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0cffb0 con 0x7f9e880051d0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8800c120 con 0x7f9e880039c0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8800c2e0 con 0x7f9e880047c0 2024-01-19T04:56:58.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.374+0000 7f9ea2219640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e880051d0 0x7f9e8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea1a18640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880039c0 0x7f9e88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea1217640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e880047c0 0x7f9e88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea2219640 1 -- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e880047c0 msgr2=0x7f9e88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea2219640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e880047c0 0x7f9e88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea2219640 1 -- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880039c0 msgr2=0x7f9e88003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea2219640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880039c0 0x7f9e88003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea2219640 1 -- 172.21.15.31:0/1785857830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e88003520 con 0x7f9e880051d0 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea2219640 1 -- 172.21.15.31:0/1785857830 reap_dead start 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.375+0000 7f9ea2219640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e880051d0 0x7f9e8800b980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c06d790 tx=0x7f9e880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72020ee08236eb95 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94004020 con 0x7f9e880051d0 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94004180 con 0x7f9e880051d0 2024-01-19T04:56:58.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea0a16640 1 -- 172.21.15.31:0/1785857830 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940044f0 con 0x7f9e880051d0 2024-01-19T04:56:58.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1785857830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e880051d0 msgr2=0x7f9e8800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e880051d0 0x7f9e8800b980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c06d790 tx=0x7f9e880057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1785857830 shutdown_connections 2024-01-19T04:56:58.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1785857830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e880051d0 0x7f9e8800b980 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1785857830 >> 172.21.15.31:0/1785857830 conn(0x7f9e9c0d1100 msgr2=0x7f9e9c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1785857830 shutdown_connections 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.376+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1785857830 wait complete. 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.377+0000 7f9ea3c7c640 1 Processor -- start 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.377+0000 7f9ea3c7c640 1 -- start start 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea3c7c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880051d0 0x7f9e9c14c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea3c7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e9c0b5e90 0x7f9e9c0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea3c7c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e9c0c5a80 0x7f9e9c0c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea3c7c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b6850 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea3c7c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0c6440 con 0x7f9e880051d0 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea3c7c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0c4eb0 con 0x7f9e9c0c5a80 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea1217640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e9c0b5e90 0x7f9e9c0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea1217640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e9c0b5e90 0x7f9e9c0b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40082/0 (socket says 172.21.15.31:40082) 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.378+0000 7f9ea1217640 1 -- 172.21.15.31:0/1642517712 learned_addr learned my addr 172.21.15.31:0/1642517712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:58.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea1a18640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880051d0 0x7f9e9c14c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea2219640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e9c0c5a80 0x7f9e9c0c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea1217640 1 -- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e9c0c5a80 msgr2=0x7f9e9c0c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea1217640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e9c0c5a80 0x7f9e9c0c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea1217640 1 -- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880051d0 msgr2=0x7f9e9c14c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea1217640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880051d0 0x7f9e9c14c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea1217640 1 -- 172.21.15.31:0/1642517712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c09b770 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea2219640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e9c0c5a80 0x7f9e9c0c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea1217640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e9c0b5e90 0x7f9e9c0b6310 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00f620 tx=0x7f9e8c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9e927fc640 1 -- 172.21.15.31:0/1642517712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0052e0 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9e927fc640 1 -- 172.21.15.31:0/1642517712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c005440 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c14ca90 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9e927fc640 1 -- 172.21.15.31:0/1642517712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c003810 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.379+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c09ae40 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.380+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c09b900 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.381+0000 7f9e927fc640 1 -- 172.21.15.31:0/1642517712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9e8c003970 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.381+0000 7f9e927fc640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9e8006af00 0x7f9e8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.381+0000 7f9e927fc640 1 -- 172.21.15.31:0/1642517712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9e8c0895b0 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.381+0000 7f9ea1a18640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9e8006af00 0x7f9e8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.382+0000 7f9ea1a18640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9e8006af00 0x7f9e8006d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001550 tx=0x7f9e98007ef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.386+0000 7f9e927fc640 1 -- 172.21.15.31:0/1642517712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9e8c0542f0 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.564+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e9c074900 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.564+0000 7f9e927fc640 1 -- 172.21.15.31:0/1642517712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e8c057910 con 0x7f9e9c0b5e90 2024-01-19T04:56:58.565 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9e8006af00 msgr2=0x7f9e8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9e8006af00 0x7f9e8006d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001550 tx=0x7f9e98007ef0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e9c0b5e90 msgr2=0x7f9e9c0b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e9c0b5e90 0x7f9e9c0b6310 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00f620 tx=0x7f9e8c00fae0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 shutdown_connections 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9e8006af00 0x7f9e8006d3c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9e9c0c5a80 0x7f9e9c0c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e9c0b5e90 0x7f9e9c0b6310 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 --2- 172.21.15.31:0/1642517712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e880051d0 0x7f9e9c14c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 >> 172.21.15.31:0/1642517712 conn(0x7f9e9c0d1100 msgr2=0x7f9e9c15ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 shutdown_connections 2024-01-19T04:56:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.566+0000 7f9ea3c7c640 1 -- 172.21.15.31:0/1642517712 wait complete. 2024-01-19T04:56:58.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-19T04:56:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.681+0000 7f7948505640 1 Processor -- start 2024-01-19T04:56:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.681+0000 7f7948505640 1 -- start start 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.681+0000 7f7948505640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f79400ce090 0x7f79400d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.681+0000 7f7948505640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400cce50 con 0x7f79400cd2e0 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f7948505640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400cc240 con 0x7f794015b6b0 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f7948505640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400d03b0 con 0x7f79400ce090 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f7946aa2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f79400cd2e0 0x7f79400b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34032/0 (socket says 172.21.15.31:34032) 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f7946aa2640 1 -- 172.21.15.31:0/1669644366 learned_addr learned my addr 172.21.15.31:0/1669644366 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f7945aa0640 1 --2- 172.21.15.31:0/1669644366 >> v2:172.21.15.137:3300/0 conn(0x7f79400ce090 0x7f79400d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564362783 0 0) 0x7f79400cce50 con 0x7f79400cd2e0 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f792c003520 con 0x7f79400cd2e0 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 783306714 0 0) 0x7f792c003520 con 0x7f79400cd2e0 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 >> v2:172.21.15.137:3300/0 conn(0x7f79400ce090 msgr2=0x7f79400d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 --2- 172.21.15.31:0/1669644366 >> v2:172.21.15.137:3300/0 conn(0x7f79400ce090 0x7f79400d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 >> v1:172.21.15.137:6789/0 conn(0x7f794015b6b0 legacy=0x7f79400cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400cc520 con 0x7f79400cd2e0 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f79380019f0 con 0x7f79400cd2e0 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 >> v1:172.21.15.31:6789/0 conn(0x7f79400cd2e0 legacy=0x7f79400b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f792c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f792c0047c0 0x7f792c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c005180 0x7f792c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400d03b0 con 0x7f792c003970 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c00c020 con 0x7f792c005180 2024-01-19T04:56:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c00c1e0 con 0x7f792c0047c0 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.682+0000 7f79462a1640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f792c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f7946aa2640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c005180 0x7f792c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f7945aa0640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f792c0047c0 0x7f792c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f79462a1640 1 -- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f792c0047c0 msgr2=0x7f792c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f79462a1640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f792c0047c0 0x7f792c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f79462a1640 1 -- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c005180 msgr2=0x7f792c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f79462a1640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f792c005180 0x7f792c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f79462a1640 1 -- 172.21.15.31:0/1669644366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c003520 con 0x7f792c003970 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f7946aa2640 1 -- 172.21.15.31:0/1669644366 reap_dead start 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f7945aa0640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f792c0047c0 0x7f792c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.683+0000 7f79462a1640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f792c003d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f793c001770 tx=0x7f792c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64eb8f393663413d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0052d0 con 0x7f792c003970 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c02e050 con 0x7f792c003970 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f794529f640 1 -- 172.21.15.31:0/1669644366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c003640 con 0x7f792c003970 2024-01-19T04:56:58.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f7948505640 1 -- 172.21.15.31:0/1669644366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 msgr2=0x7f792c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f7948505640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f792c003d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f793c001770 tx=0x7f792c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f7948505640 1 -- 172.21.15.31:0/1669644366 shutdown_connections 2024-01-19T04:56:58.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f7948505640 1 --2- 172.21.15.31:0/1669644366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f792c003d90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f7948505640 1 -- 172.21.15.31:0/1669644366 >> 172.21.15.31:0/1669644366 conn(0x7f79400d1100 msgr2=0x7f79400d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:58.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f7948505640 1 -- 172.21.15.31:0/1669644366 shutdown_connections 2024-01-19T04:56:58.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.684+0000 7f7948505640 1 -- 172.21.15.31:0/1669644366 wait complete. 2024-01-19T04:56:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.685+0000 7f7948505640 1 Processor -- start 2024-01-19T04:56:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.685+0000 7f7948505640 1 -- start start 2024-01-19T04:56:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f7948505640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f79400b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f7948505640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79400b3cc0 0x7f79400c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f7948505640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7940148ff0 0x7f7940149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f7948505640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b5a60 con 0x7f792c003970 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f7948505640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b4480 con 0x7f79400b3cc0 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f7948505640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b4760 con 0x7f7940148ff0 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f79462a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f79400b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f79462a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f79400b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40120/0 (socket says 172.21.15.31:40120) 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f79462a1640 1 -- 172.21.15.31:0/3840714706 learned_addr learned my addr 172.21.15.31:0/3840714706 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f79462a1640 1 -- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7940148ff0 msgr2=0x7f7940149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f79462a1640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7940148ff0 0x7f7940149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.686+0000 7f79462a1640 1 -- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79400b3cc0 msgr2=0x7f79400c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f79462a1640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79400b3cc0 0x7f79400c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f79462a1640 1 -- 172.21.15.31:0/3840714706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400b4a40 con 0x7f792c003970 2024-01-19T04:56:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f79462a1640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f79400b5520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f793c005040 tx=0x7f793c001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f7936ffd640 1 -- 172.21.15.31:0/3840714706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c007a80 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f7936ffd640 1 -- 172.21.15.31:0/3840714706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c006e20 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79400b4d20 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f7936ffd640 1 -- 172.21.15.31:0/3840714706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c007450 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.687+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79400b4140 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.688+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79400cec30 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.688+0000 7f7936ffd640 1 -- 172.21.15.31:0/3840714706 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f793c0075b0 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.689+0000 7f7936ffd640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792406aed0 0x7f792406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.689+0000 7f7936ffd640 1 -- 172.21.15.31:0/3840714706 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f793c0ab540 con 0x7f792c003970 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.689+0000 7f7945aa0640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792406aed0 0x7f792406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.689+0000 7f7945aa0640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792406aed0 0x7f792406d390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f79400c7cc0 tx=0x7f7930008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.693+0000 7f7936ffd640 1 -- 172.21.15.31:0/3840714706 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f793c079ec0 con 0x7f792c003970 2024-01-19T04:56:58.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.872+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f7940074900 con 0x7f792c003970 2024-01-19T04:56:58.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.873+0000 7f7936ffd640 1 -- 172.21.15.31:0/3840714706 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7f793c079a50 con 0x7f792c003970 2024-01-19T04:56:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:56:58.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792406aed0 msgr2=0x7f792406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792406aed0 0x7f792406d390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f79400c7cc0 tx=0x7f7930008040 comp rx=0 tx=0).stop 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 msgr2=0x7f79400b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f79400b5520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f793c005040 tx=0x7f793c001170 comp rx=0 tx=0).stop 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 shutdown_connections 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f792406aed0 0x7f792406d390 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7940148ff0 0x7f7940149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79400b3cc0 0x7f79400c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 --2- 172.21.15.31:0/3840714706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792c003970 0x7f79400b5520 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 >> 172.21.15.31:0/3840714706 conn(0x7f79400d1100 msgr2=0x7f794006fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 shutdown_connections 2024-01-19T04:56:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.875+0000 7f7948505640 1 -- 172.21.15.31:0/3840714706 wait complete. 2024-01-19T04:56:58.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:58.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.986+0000 7f9c9f4ad640 1 Processor -- start 2024-01-19T04:56:58.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.987+0000 7f9c9f4ad640 1 -- start start 2024-01-19T04:56:58.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.987+0000 7f9c9f4ad640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c980b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.987+0000 7f9c9f4ad640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c980d02f0 0x7f9c980b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.987+0000 7f9c9f4ad640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c980d22b0 con 0x7f9c980cf150 2024-01-19T04:56:58.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.987+0000 7f9c9f4ad640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c980d41c0 con 0x7f9c980d02f0 2024-01-19T04:56:58.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.987+0000 7f9c9f4ad640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980d1980 con 0x7f9c98161ee0 2024-01-19T04:56:58.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9da4a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c980d02f0 0x7f9c980b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9da4a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c980d02f0 0x7f9c980b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40124/0 (socket says 172.21.15.31:40124) 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9da4a640 1 -- 172.21.15.31:0/2151213354 learned_addr learned my addr 172.21.15.31:0/2151213354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9d249640 1 --2- 172.21.15.31:0/2151213354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c980b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9da4a640 1 -- 172.21.15.31:0/2151213354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 msgr2=0x7f9c980b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9da4a640 1 --2- 172.21.15.31:0/2151213354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c980b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9da4a640 1 -- 172.21.15.31:0/2151213354 >> v1:172.21.15.137:6789/0 conn(0x7f9c980cf150 legacy=0x7f9c980cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9da4a640 1 -- 172.21.15.31:0/2151213354 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980d1c60 con 0x7f9c980d02f0 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.988+0000 7f9c9d249640 1 --2- 172.21.15.31:0/2151213354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c980b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9da4a640 1 --2- 172.21.15.31:0/2151213354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c980b63a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009e50 tx=0x7f9c8802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4962da2ecc7152 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c97fff640 1 -- 172.21.15.31:0/2151213354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8802ca70 con 0x7f9c980d02f0 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c97fff640 1 -- 172.21.15.31:0/2151213354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8802cbd0 con 0x7f9c980d02f0 2024-01-19T04:56:58.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c97fff640 1 -- 172.21.15.31:0/2151213354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880343f0 con 0x7f9c980d02f0 2024-01-19T04:56:58.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/2151213354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 msgr2=0x7f9c980b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/2151213354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c980b63a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009e50 tx=0x7f9c8802bed0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/2151213354 shutdown_connections 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/2151213354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c980b63a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/2151213354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c980b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/2151213354 >> 172.21.15.31:0/2151213354 conn(0x7f9c9809c680 msgr2=0x7f9c980d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/2151213354 shutdown_connections 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.989+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/2151213354 wait complete. 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.990+0000 7f9c9f4ad640 1 Processor -- start 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9f4ad640 1 -- start start 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9f4ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c980cf150 0x7f9c980c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9f4ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c9814c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9f4ad640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c9814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9f4ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c981ea930 con 0x7f9c980d02f0 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9f4ad640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9814c030 con 0x7f9c980cf150 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9f4ad640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9814b420 con 0x7f9c98161ee0 2024-01-19T04:56:58.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9ca48640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c9814c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9ca48640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c9814c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40132/0 (socket says 172.21.15.31:40132) 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.991+0000 7f9c9ca48640 1 -- 172.21.15.31:0/207114326 learned_addr learned my addr 172.21.15.31:0/207114326 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9d249640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c980cf150 0x7f9c980c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9ca48640 1 -- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 msgr2=0x7f9c9814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9da4a640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c9814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9ca48640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c9814ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9ca48640 1 -- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c980cf150 msgr2=0x7f9c980c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9ca48640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c980cf150 0x7f9c980c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9ca48640 1 -- 172.21.15.31:0/207114326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88009b30 con 0x7f9c980d02f0 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9d249640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c980cf150 0x7f9c980c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9da4a640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c9814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9ca48640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c9814c4c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c400 tx=0x7f9c8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c95ffb640 1 -- 172.21.15.31:0/207114326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c011790 con 0x7f9c980d02f0 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c95ffb640 1 -- 172.21.15.31:0/207114326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c011930 con 0x7f9c980d02f0 2024-01-19T04:56:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9814b700 con 0x7f9c980d02f0 2024-01-19T04:56:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c95ffb640 1 -- 172.21.15.31:0/207114326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c011ce0 con 0x7f9c980d02f0 2024-01-19T04:56:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.992+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9814bcc0 con 0x7f9c980d02f0 2024-01-19T04:56:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.993+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c98074a80 con 0x7f9c980d02f0 2024-01-19T04:56:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.993+0000 7f9c95ffb640 1 -- 172.21.15.31:0/207114326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9c8c01c460 con 0x7f9c980d02f0 2024-01-19T04:56:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.994+0000 7f9c95ffb640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c8406aed0 0x7f9c8406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.994+0000 7f9c95ffb640 1 -- 172.21.15.31:0/207114326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9c8c0897c0 con 0x7f9c980d02f0 2024-01-19T04:56:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.994+0000 7f9c9d249640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c8406aed0 0x7f9c8406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.995+0000 7f9c9d249640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c8406aed0 0x7f9c8406d390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:58.998+0000 7f9c95ffb640 1 -- 172.21.15.31:0/207114326 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9c8c054470 con 0x7f9c980d02f0 2024-01-19T04:56:59.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.214+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c980696e0 con 0x7f9c980d02f0 2024-01-19T04:56:59.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.214+0000 7f9c95ffb640 1 -- 172.21.15.31:0/207114326 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f9c8c057a90 con 0x7f9c980d02f0 2024-01-19T04:56:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:59.215 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:56:25.278275+0000","modified":"2024-01-19T04:56:32.866830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5259":{"gid":5259,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/968278604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":968278604},{"type":"v1","addr":"172.21.15.31:6835","nonce":968278604}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-19T04:56:59.215 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c8406aed0 msgr2=0x7f9c8406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c8406aed0 0x7f9c8406d390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90008040 comp rx=0 tx=0).stop 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 msgr2=0x7f9c9814c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c9814c4c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c400 tx=0x7f9c8c00c8c0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 shutdown_connections 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c8406aed0 0x7f9c8406d390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c98161ee0 0x7f9c9814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c980d02f0 0x7f9c9814c4c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 --2- 172.21.15.31:0/207114326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c980cf150 0x7f9c980c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 >> 172.21.15.31:0/207114326 conn(0x7f9c9809c680 msgr2=0x7f9c980b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 shutdown_connections 2024-01-19T04:56:59.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.216+0000 7f9c9f4ad640 1 -- 172.21.15.31:0/207114326 wait complete. 2024-01-19T04:56:59.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:56:59.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.327+0000 7f17290d5640 1 Processor -- start 2024-01-19T04:56:59.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.327+0000 7f17290d5640 1 -- start start 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.328+0000 7f17290d5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f172406baa0 0x7f17240d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.328+0000 7f17290d5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f17240cda90 0x7f17240cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.328+0000 7f17290d5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172415d920 con 0x7f172415d540 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.328+0000 7f17290d5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17240ce440 con 0x7f17240cda90 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.328+0000 7f17290d5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172406c040 con 0x7f172406baa0 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.328+0000 7f1722d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f172415d540 0x7f17240cb760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34042/0 (socket says 172.21.15.31:34042) 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.328+0000 7f1722d9d640 1 -- 172.21.15.31:0/2868163218 learned_addr learned my addr 172.21.15.31:0/2868163218 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f172259c640 1 --2- 172.21.15.31:0/2868163218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f172406baa0 0x7f17240d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f172359e640 1 --2- 172.21.15.31:0/2868163218 >> v2:172.21.15.137:3300/0 conn(0x7f17240cda90 0x7f17240cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f1721d9b640 1 -- 172.21.15.31:0/2868163218 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573879798 0 0) 0x7f172415d920 con 0x7f172415d540 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f1721d9b640 1 -- 172.21.15.31:0/2868163218 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f170c003520 con 0x7f172415d540 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f1721d9b640 1 -- 172.21.15.31:0/2868163218 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2020642431 0 0) 0x7f170c003520 con 0x7f172415d540 2024-01-19T04:56:59.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f172259c640 1 -- 172.21.15.31:0/2868163218 >> v2:172.21.15.137:3300/0 conn(0x7f17240cda90 msgr2=0x7f17240cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f172259c640 1 --2- 172.21.15.31:0/2868163218 >> v2:172.21.15.137:3300/0 conn(0x7f17240cda90 0x7f17240cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f172259c640 1 -- 172.21.15.31:0/2868163218 >> v1:172.21.15.31:6789/0 conn(0x7f172415d540 legacy=0x7f17240cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.329+0000 7f172259c640 1 -- 172.21.15.31:0/2868163218 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172406c320 con 0x7f172406baa0 2024-01-19T04:56:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f172259c640 1 --2- 172.21.15.31:0/2868163218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f172406baa0 0x7f17240d0a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17180097a0 tx=0x7f171802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1ba3cc64cfaf28 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f1721d9b640 1 -- 172.21.15.31:0/2868163218 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c001670 con 0x7f172406baa0 2024-01-19T04:56:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f1721d9b640 1 -- 172.21.15.31:0/2868163218 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171802e050 con 0x7f172406baa0 2024-01-19T04:56:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f1721d9b640 1 -- 172.21.15.31:0/2868163218 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718030710 con 0x7f172406baa0 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 -- 172.21.15.31:0/2868163218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f172406baa0 msgr2=0x7f17240d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 --2- 172.21.15.31:0/2868163218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f172406baa0 0x7f17240d0a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17180097a0 tx=0x7f171802b790 comp rx=0 tx=0).stop 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 -- 172.21.15.31:0/2868163218 shutdown_connections 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 --2- 172.21.15.31:0/2868163218 >> v2:172.21.15.137:3300/0 conn(0x7f17240cda90 0x7f17240cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 --2- 172.21.15.31:0/2868163218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f172406baa0 0x7f17240d0a70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 -- 172.21.15.31:0/2868163218 >> 172.21.15.31:0/2868163218 conn(0x7f17240b2c60 msgr2=0x7f17240b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 -- 172.21.15.31:0/2868163218 shutdown_connections 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.330+0000 7f17290d5640 1 -- 172.21.15.31:0/2868163218 wait complete. 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.332+0000 7f17290d5640 1 Processor -- start 2024-01-19T04:56:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.332+0000 7f17290d5640 1 -- start start 2024-01-19T04:56:59.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f17290d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172406baa0 0x7f17240c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f17290d5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17240cda90 0x7f1724158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f17290d5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172415d540 0x7f17240ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f17290d5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240ca210 con 0x7f172406baa0 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f17290d5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240c9600 con 0x7f172415d540 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f17290d5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240c98e0 con 0x7f17240cda90 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172406baa0 0x7f17240c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172406baa0 0x7f17240c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40148/0 (socket says 172.21.15.31:40148) 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 -- 172.21.15.31:0/2385324966 learned_addr learned my addr 172.21.15.31:0/2385324966 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f172359e640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172415d540 0x7f17240ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f172259c640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17240cda90 0x7f1724158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 -- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17240cda90 msgr2=0x7f1724158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17240cda90 0x7f1724158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 -- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172415d540 msgr2=0x7f17240ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172415d540 0x7f17240ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 -- 172.21.15.31:0/2385324966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718009400 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f1722d9d640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172406baa0 0x7f17240c5690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c750 tx=0x7f171c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f170b7fe640 1 -- 172.21.15.31:0/2385324966 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c005ad0 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f170b7fe640 1 -- 172.21.15.31:0/2385324966 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c005c30 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17240c9bc0 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.333+0000 7f170b7fe640 1 -- 172.21.15.31:0/2385324966 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c007070 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.334+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17240c5bd0 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.334+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1724074a80 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.335+0000 7f170b7fe640 1 -- 172.21.15.31:0/2385324966 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f171c0071d0 con 0x7f172406baa0 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.335+0000 7f170b7fe640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f171006aed0 0x7f171006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.335+0000 7f170b7fe640 1 -- 172.21.15.31:0/2385324966 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f171c0892d0 con 0x7f172406baa0 2024-01-19T04:56:59.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.336+0000 7f172259c640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f171006aed0 0x7f171006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.336+0000 7f172259c640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f171006aed0 0x7f171006d390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1718034000 tx=0x7f171802f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.340+0000 7f170b7fe640 1 -- 172.21.15.31:0/2385324966 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f171c054150 con 0x7f172406baa0 2024-01-19T04:56:59.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.519+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17240696e0 con 0x7f172406baa0 2024-01-19T04:56:59.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.521+0000 7f170b7fe640 1 -- 172.21.15.31:0/2385324966 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f171c057770 con 0x7f172406baa0 2024-01-19T04:56:59.521 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:59.521 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:56:59.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f171006aed0 msgr2=0x7f171006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f171006aed0 0x7f171006d390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1718034000 tx=0x7f171802f000 comp rx=0 tx=0).stop 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172406baa0 msgr2=0x7f17240c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172406baa0 0x7f17240c5690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c750 tx=0x7f171c00cc10 comp rx=0 tx=0).stop 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 shutdown_connections 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f171006aed0 0x7f171006d390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172415d540 0x7f17240ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17240cda90 0x7f1724158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 --2- 172.21.15.31:0/2385324966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172406baa0 0x7f17240c5690 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 >> 172.21.15.31:0/2385324966 conn(0x7f17240b2c60 msgr2=0x7f17240b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 shutdown_connections 2024-01-19T04:56:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.523+0000 7f17290d5640 1 -- 172.21.15.31:0/2385324966 wait complete. 2024-01-19T04:56:59.534 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-19T04:56:59.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:59.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.635+0000 7fec80d9a640 1 Processor -- start 2024-01-19T04:56:59.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.635+0000 7fec80d9a640 1 -- start start 2024-01-19T04:56:59.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec80d9a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec7c160bb0 0x7fec7c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec80d9a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fec7c0d1600 0x7fec7c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec80d9a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c0d06b0 con 0x7fec7c0cfd10 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec80d9a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c0b66b0 con 0x7fec7c160bb0 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec80d9a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c0d1b90 con 0x7fec7c0d1600 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec7c160bb0 0x7fec7c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec7c160bb0 0x7fec7c0b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52010/0 (socket says 172.21.15.31:52010) 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7a59c640 1 -- 172.21.15.31:0/2802819676 learned_addr learned my addr 172.21.15.31:0/2802819676 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7ad9d640 1 --2- 172.21.15.31:0/2802819676 >> v2:172.21.15.137:3300/0 conn(0x7fec7c0d1600 0x7fec7c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7a59c640 1 -- 172.21.15.31:0/2802819676 >> v2:172.21.15.137:3300/0 conn(0x7fec7c0d1600 msgr2=0x7fec7c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7a59c640 1 --2- 172.21.15.31:0/2802819676 >> v2:172.21.15.137:3300/0 conn(0x7fec7c0d1600 0x7fec7c0d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7a59c640 1 -- 172.21.15.31:0/2802819676 >> v1:172.21.15.137:6789/0 conn(0x7fec7c0cfd10 legacy=0x7fec7c0d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.636+0000 7fec7a59c640 1 -- 172.21.15.31:0/2802819676 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0d1e70 con 0x7fec7c160bb0 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.637+0000 7fec7a59c640 1 --2- 172.21.15.31:0/2802819676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c0b6170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0b93b0 tx=0x7fec6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce8c03755a54c08 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.637+0000 7fec7959a640 1 -- 172.21.15.31:0/2802819676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c031870 con 0x7fec7c160bb0 2024-01-19T04:56:59.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.637+0000 7fec7959a640 1 -- 172.21.15.31:0/2802819676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c0319d0 con 0x7fec7c160bb0 2024-01-19T04:56:59.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.637+0000 7fec7959a640 1 -- 172.21.15.31:0/2802819676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c031d60 con 0x7fec7c160bb0 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 -- 172.21.15.31:0/2802819676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 msgr2=0x7fec7c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 --2- 172.21.15.31:0/2802819676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c0b6170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0b93b0 tx=0x7fec6c02b910 comp rx=0 tx=0).stop 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 -- 172.21.15.31:0/2802819676 shutdown_connections 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 --2- 172.21.15.31:0/2802819676 >> v2:172.21.15.137:3300/0 conn(0x7fec7c0d1600 0x7fec7c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 --2- 172.21.15.31:0/2802819676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c0b6170 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 -- 172.21.15.31:0/2802819676 >> 172.21.15.31:0/2802819676 conn(0x7fec7c09c600 msgr2=0x7fec7c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 -- 172.21.15.31:0/2802819676 shutdown_connections 2024-01-19T04:56:59.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.638+0000 7fec80d9a640 1 -- 172.21.15.31:0/2802819676 wait complete. 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.639+0000 7fec80d9a640 1 Processor -- start 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.639+0000 7fec80d9a640 1 -- start start 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec80d9a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec7c0cfd10 0x7fec7c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec80d9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec7c0d1600 0x7fec7c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec80d9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec80d9a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0c9e70 con 0x7fec7c160bb0 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec80d9a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0cb700 con 0x7fec7c0d1600 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec80d9a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0cb3d0 con 0x7fec7c0cfd10 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52020/0 (socket says 172.21.15.31:52020) 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 -- 172.21.15.31:0/630750441 learned_addr learned my addr 172.21.15.31:0/630750441 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec79d9b640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec7c0d1600 0x7fec7c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7a59c640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec7c0cfd10 0x7fec7c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 -- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec7c0cfd10 msgr2=0x7fec7c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec7c0cfd10 0x7fec7c15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 -- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec7c0d1600 msgr2=0x7fec7c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec7c0d1600 0x7fec7c0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 -- 172.21.15.31:0/630750441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0094d0 con 0x7fec7c160bb0 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.640+0000 7fec7ad9d640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c15b280 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fec74007e40 tx=0x7fec7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.641+0000 7fec6affd640 1 -- 172.21.15.31:0/630750441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74014620 con 0x7fec7c160bb0 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.641+0000 7fec6affd640 1 -- 172.21.15.31:0/630750441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400ee90 con 0x7fec7c160bb0 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.641+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c0c98b0 con 0x7fec7c160bb0 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.641+0000 7fec6affd640 1 -- 172.21.15.31:0/630750441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740149e0 con 0x7fec7c160bb0 2024-01-19T04:56:59.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.641+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c15d2d0 con 0x7fec7c160bb0 2024-01-19T04:56:59.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.642+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c0cb230 con 0x7fec7c160bb0 2024-01-19T04:56:59.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.642+0000 7fec6affd640 1 -- 172.21.15.31:0/630750441 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fec74014b40 con 0x7fec7c160bb0 2024-01-19T04:56:59.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.643+0000 7fec6affd640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fec5c06aed0 0x7fec5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.643+0000 7fec6affd640 1 -- 172.21.15.31:0/630750441 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fec74089f90 con 0x7fec7c160bb0 2024-01-19T04:56:59.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.643+0000 7fec7a59c640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fec5c06aed0 0x7fec5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.643+0000 7fec7a59c640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fec5c06aed0 0x7fec5c06d390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec6c036000 tx=0x7fec6c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.647+0000 7fec6affd640 1 -- 172.21.15.31:0/630750441 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fec740bd0d0 con 0x7fec7c160bb0 2024-01-19T04:56:59.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.860+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7c0696e0 con 0x7fec7c160bb0 2024-01-19T04:56:59.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.860+0000 7fec6affd640 1 -- 172.21.15.31:0/630750441 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fec7c0cb230 con 0x7fec7c160bb0 2024-01-19T04:56:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:56:59.861 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:56:25.278275+0000","modified":"2024-01-19T04:56:32.866830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5259":{"gid":5259,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/968278604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":968278604},{"type":"v1","addr":"172.21.15.31:6835","nonce":968278604}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-19T04:56:59.861 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fec5c06aed0 msgr2=0x7fec5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fec5c06aed0 0x7fec5c06d390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec6c036000 tx=0x7fec6c03c040 comp rx=0 tx=0).stop 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 msgr2=0x7fec7c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c15b280 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fec74007e40 tx=0x7fec7400e4b0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 shutdown_connections 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fec5c06aed0 0x7fec5c06d390 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c160bb0 0x7fec7c15b280 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec7c0d1600 0x7fec7c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 --2- 172.21.15.31:0/630750441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec7c0cfd10 0x7fec7c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 >> 172.21.15.31:0/630750441 conn(0x7fec7c09c600 msgr2=0x7fec7c06bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 shutdown_connections 2024-01-19T04:56:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.862+0000 7fec80d9a640 1 -- 172.21.15.31:0/630750441 wait complete. 2024-01-19T04:56:59.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:56:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.974+0000 7f059733c640 1 Processor -- start 2024-01-19T04:56:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.975+0000 7f059733c640 1 -- start start 2024-01-19T04:56:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.975+0000 7f059733c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05900d02f0 0x7f05900b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.975+0000 7f059733c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05900d22b0 con 0x7f05900cf150 2024-01-19T04:56:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.975+0000 7f059733c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05900d41c0 con 0x7f0590161ee0 2024-01-19T04:56:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.975+0000 7f059733c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05900d1980 con 0x7f05900d02f0 2024-01-19T04:56:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.976+0000 7f05948d7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05900d02f0 0x7f05900b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.976+0000 7f05958d9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05900cf150 0x7f05900b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42592/0 (socket says 172.21.15.31:42592) 2024-01-19T04:56:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.976+0000 7f05958d9640 1 -- 172.21.15.31:0/72126137 learned_addr learned my addr 172.21.15.31:0/72126137 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:56:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.976+0000 7f05948d7640 1 -- 172.21.15.31:0/72126137 >> v1:172.21.15.137:6789/0 conn(0x7f0590161ee0 legacy=0x7f05900cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:56:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.976+0000 7f05948d7640 1 -- 172.21.15.31:0/72126137 >> v1:172.21.15.31:6789/0 conn(0x7f05900cf150 legacy=0x7f05900b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.976+0000 7f05948d7640 1 -- 172.21.15.31:0/72126137 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900d1c60 con 0x7f05900d02f0 2024-01-19T04:56:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f05948d7640 1 --2- 172.21.15.31:0/72126137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900d02f0 0x7f05900b40d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f05800097a0 tx=0x7f058002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d7702c96af8b4c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f0587fff640 1 -- 172.21.15.31:0/72126137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580031830 con 0x7f05900d02f0 2024-01-19T04:56:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f0587fff640 1 -- 172.21.15.31:0/72126137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580031990 con 0x7f05900d02f0 2024-01-19T04:56:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f0587fff640 1 -- 172.21.15.31:0/72126137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580031d20 con 0x7f05900d02f0 2024-01-19T04:56:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f059733c640 1 -- 172.21.15.31:0/72126137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900d02f0 msgr2=0x7f05900b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f059733c640 1 --2- 172.21.15.31:0/72126137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900d02f0 0x7f05900b40d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f05800097a0 tx=0x7f058002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f059733c640 1 -- 172.21.15.31:0/72126137 shutdown_connections 2024-01-19T04:56:59.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f059733c640 1 --2- 172.21.15.31:0/72126137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900d02f0 0x7f05900b40d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f059733c640 1 -- 172.21.15.31:0/72126137 >> 172.21.15.31:0/72126137 conn(0x7f059009c680 msgr2=0x7f05900d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:56:59.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f059733c640 1 -- 172.21.15.31:0/72126137 shutdown_connections 2024-01-19T04:56:59.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.977+0000 7f059733c640 1 -- 172.21.15.31:0/72126137 wait complete. 2024-01-19T04:56:59.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.978+0000 7f059733c640 1 Processor -- start 2024-01-19T04:56:59.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.978+0000 7f059733c640 1 -- start start 2024-01-19T04:56:59.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f059733c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05900cf150 0x7f05900ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f059733c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05900d02f0 0x7f059015f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f059733c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590161ee0 0x7f059015fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f059733c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0590161100 con 0x7f0590161ee0 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f059733c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0590160b40 con 0x7f05900cf150 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f059733c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0590160e20 con 0x7f05900d02f0 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f05958d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590161ee0 0x7f059015fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f05958d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590161ee0 0x7f059015fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52038/0 (socket says 172.21.15.31:52038) 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f05958d9640 1 -- 172.21.15.31:0/2595323106 learned_addr learned my addr 172.21.15.31:0/2595323106 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f05958d9640 1 -- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05900d02f0 msgr2=0x7f059015f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.979+0000 7f05958d9640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05900d02f0 0x7f059015f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f05958d9640 1 -- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05900cf150 msgr2=0x7f05900ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f05958d9640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05900cf150 0x7f05900ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:56:59.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f05958d9640 1 -- 172.21.15.31:0/2595323106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0580009400 con 0x7f0590161ee0 2024-01-19T04:56:59.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f05958d9640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590161ee0 0x7f059015fe30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f058800caa0 tx=0x7f058800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f0585ffb640 1 -- 172.21.15.31:0/2595323106 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058800ac00 con 0x7f0590161ee0 2024-01-19T04:56:59.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f0585ffb640 1 -- 172.21.15.31:0/2595323106 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058800ad60 con 0x7f0590161ee0 2024-01-19T04:56:59.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059015ebf0 con 0x7f0590161ee0 2024-01-19T04:56:59.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f0585ffb640 1 -- 172.21.15.31:0/2595323106 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588012600 con 0x7f0590161ee0 2024-01-19T04:56:59.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.980+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059015f930 con 0x7f0590161ee0 2024-01-19T04:56:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.981+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0590074a80 con 0x7f0590161ee0 2024-01-19T04:56:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.981+0000 7f0585ffb640 1 -- 172.21.15.31:0/2595323106 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0588007070 con 0x7f0590161ee0 2024-01-19T04:56:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.982+0000 7f0585ffb640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f057406aed0 0x7f057406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:56:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.982+0000 7f0585ffb640 1 -- 172.21.15.31:0/2595323106 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0588089660 con 0x7f0590161ee0 2024-01-19T04:56:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.982+0000 7f05950d8640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f057406aed0 0x7f057406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:56:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.982+0000 7f05950d8640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f057406aed0 0x7f057406d390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f058c006fd0 tx=0x7f058c006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:56:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:56:59.986+0000 7f0585ffb640 1 -- 172.21.15.31:0/2595323106 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f05880543c0 con 0x7f0590161ee0 2024-01-19T04:57:00.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.201+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f059015f110 con 0x7f0590161ee0 2024-01-19T04:57:00.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.202+0000 7f0585ffb640 1 -- 172.21.15.31:0/2595323106 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f059015f110 con 0x7f0590161ee0 2024-01-19T04:57:00.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:00.202 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:56:25.278275+0000","modified":"2024-01-19T04:56:32.866830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5259":{"gid":5259,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/968278604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":968278604},{"type":"v1","addr":"172.21.15.31:6835","nonce":968278604}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-19T04:57:00.203 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f057406aed0 msgr2=0x7f057406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f057406aed0 0x7f057406d390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f058c006fd0 tx=0x7f058c006bc0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590161ee0 msgr2=0x7f059015fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590161ee0 0x7f059015fe30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f058800caa0 tx=0x7f058800cf60 comp rx=0 tx=0).stop 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 shutdown_connections 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f057406aed0 0x7f057406d390 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590161ee0 0x7f059015fe30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05900d02f0 0x7f059015f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 --2- 172.21.15.31:0/2595323106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05900cf150 0x7f05900ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 >> 172.21.15.31:0/2595323106 conn(0x7f059009c680 msgr2=0x7f05900b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 shutdown_connections 2024-01-19T04:57:00.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.204+0000 7f059733c640 1 -- 172.21.15.31:0/2595323106 wait complete. 2024-01-19T04:57:00.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.315+0000 7f3ea2173640 1 Processor -- start 2024-01-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.315+0000 7f3ea2173640 1 -- start start 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3ea2173640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e9c06e8d0 0x7f3e9c0d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3ea2173640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c0d0040 con 0x7f3e9c0cee00 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3ea2173640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c0cd500 con 0x7f3e9c06ecb0 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3ea2173640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cb800 con 0x7f3e9c06e8d0 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3e9b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e9c06e8d0 0x7f3e9c0d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3e9affd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e9c0cee00 0x7f3e9c15c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42606/0 (socket says 172.21.15.31:42606) 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3e9b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e9c06e8d0 0x7f3e9c0d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52054/0 (socket says 172.21.15.31:52054) 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.316+0000 7f3e9affd640 1 -- 172.21.15.31:0/2249916116 learned_addr learned my addr 172.21.15.31:0/2249916116 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.317+0000 7f3e9b7fe640 1 -- 172.21.15.31:0/2249916116 >> v1:172.21.15.137:6789/0 conn(0x7f3e9c06ecb0 legacy=0x7f3e9c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.317+0000 7f3e9b7fe640 1 -- 172.21.15.31:0/2249916116 >> v1:172.21.15.31:6789/0 conn(0x7f3e9c0cee00 legacy=0x7f3e9c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.317+0000 7f3e9b7fe640 1 -- 172.21.15.31:0/2249916116 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0cbae0 con 0x7f3e9c06e8d0 2024-01-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.317+0000 7f3e9a7fc640 1 -- 172.21.15.31:0/2249916116 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148729525 0 0) 0x7f3e9c0d0040 con 0x7f3e9c0cee00 2024-01-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.317+0000 7f3e9b7fe640 1 --2- 172.21.15.31:0/2249916116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c06e8d0 0x7f3e9c0d08a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0097a0 tx=0x7f3e8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82569aacfe318e3f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3e9a7fc640 1 -- 172.21.15.31:0/2249916116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c02e070 con 0x7f3e9c06e8d0 2024-01-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3e9a7fc640 1 -- 172.21.15.31:0/2249916116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c0315a0 con 0x7f3e9c06e8d0 2024-01-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3e9a7fc640 1 -- 172.21.15.31:0/2249916116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c031930 con 0x7f3e9c06e8d0 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3ea2173640 1 -- 172.21.15.31:0/2249916116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c06e8d0 msgr2=0x7f3e9c0d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2249916116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c06e8d0 0x7f3e9c0d08a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0097a0 tx=0x7f3e8c02b840 comp rx=0 tx=0).stop 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3ea2173640 1 -- 172.21.15.31:0/2249916116 shutdown_connections 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2249916116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c06e8d0 0x7f3e9c0d08a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3ea2173640 1 -- 172.21.15.31:0/2249916116 >> 172.21.15.31:0/2249916116 conn(0x7f3e9c0b2c60 msgr2=0x7f3e9c0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3ea2173640 1 -- 172.21.15.31:0/2249916116 shutdown_connections 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.318+0000 7f3ea2173640 1 -- 172.21.15.31:0/2249916116 wait complete. 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.319+0000 7f3ea2173640 1 Processor -- start 2024-01-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.319+0000 7f3ea2173640 1 -- start start 2024-01-19T04:57:00.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3ea2173640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e9c06e8d0 0x7f3e9c1486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3ea2173640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e9c06ecb0 0x7f3e9c0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3ea2173640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0cee00 0x7f3e9c0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3ea2173640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1f2690 con 0x7f3e9c0cee00 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3ea2173640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cace0 con 0x7f3e9c06ecb0 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3ea2173640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cafc0 con 0x7f3e9c06e8d0 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3e9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0cee00 0x7f3e9c0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3e9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0cee00 0x7f3e9c0ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52066/0 (socket says 172.21.15.31:52066) 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3e9bfff640 1 -- 172.21.15.31:0/2864387224 learned_addr learned my addr 172.21.15.31:0/2864387224 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3e9b7fe640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e9c06e8d0 0x7f3e9c1486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.320+0000 7f3e9affd640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e9c06ecb0 0x7f3e9c0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e9bfff640 1 -- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e9c06e8d0 msgr2=0x7f3e9c1486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e9bfff640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e9c06e8d0 0x7f3e9c1486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e9bfff640 1 -- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e9c06ecb0 msgr2=0x7f3e9c0ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e9bfff640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e9c06ecb0 0x7f3e9c0ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e9bfff640 1 -- 172.21.15.31:0/2864387224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c009400 con 0x7f3e9c0cee00 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e9affd640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e9c06ecb0 0x7f3e9c0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:00.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e9bfff640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0cee00 0x7f3e9c0ca6d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400b970 tx=0x7f3e9400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e83fff640 1 -- 172.21.15.31:0/2864387224 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400cb40 con 0x7f3e9c0cee00 2024-01-19T04:57:00.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e83fff640 1 -- 172.21.15.31:0/2864387224 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9400cca0 con 0x7f3e9c0cee00 2024-01-19T04:57:00.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c0cb2a0 con 0x7f3e9c0cee00 2024-01-19T04:57:00.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3e83fff640 1 -- 172.21.15.31:0/2864387224 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94014680 con 0x7f3e9c0cee00 2024-01-19T04:57:00.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.321+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c0c5f70 con 0x7f3e9c0cee00 2024-01-19T04:57:00.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.322+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c074a80 con 0x7f3e9c0cee00 2024-01-19T04:57:00.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.322+0000 7f3e83fff640 1 -- 172.21.15.31:0/2864387224 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3e9401d3b0 con 0x7f3e9c0cee00 2024-01-19T04:57:00.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.323+0000 7f3e83fff640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e8806aed0 0x7f3e8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.323+0000 7f3e83fff640 1 -- 172.21.15.31:0/2864387224 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f3e94089600 con 0x7f3e9c0cee00 2024-01-19T04:57:00.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.323+0000 7f3e9b7fe640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e8806aed0 0x7f3e8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.324+0000 7f3e9b7fe640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e8806aed0 0x7f3e8806d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c036000 tx=0x7f3e8c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.327+0000 7f3e83fff640 1 -- 172.21.15.31:0/2864387224 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3e94054460 con 0x7f3e9c0cee00 2024-01-19T04:57:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.506+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e9c0c6190 con 0x7f3e9c0cee00 2024-01-19T04:57:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.507+0000 7f3e83fff640 1 -- 172.21.15.31:0/2864387224 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f3e94057a80 con 0x7f3e9c0cee00 2024-01-19T04:57:00.508 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:56:34.518933+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:56:23.785927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-19T04:56:24.240600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-01-19T04:56:32.396495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e8806aed0 msgr2=0x7f3e8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e8806aed0 0x7f3e8806d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c036000 tx=0x7f3e8c03d040 comp rx=0 tx=0).stop 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0cee00 msgr2=0x7f3e9c0ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0cee00 0x7f3e9c0ca6d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400b970 tx=0x7f3e9400be30 comp rx=0 tx=0).stop 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 shutdown_connections 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e8806aed0 0x7f3e8806d390 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0cee00 0x7f3e9c0ca6d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e9c06ecb0 0x7f3e9c0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 --2- 172.21.15.31:0/2864387224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e9c06e8d0 0x7f3e9c1486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 >> 172.21.15.31:0/2864387224 conn(0x7f3e9c0b2c60 msgr2=0x7f3e9c073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 shutdown_connections 2024-01-19T04:57:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.509+0000 7f3ea2173640 1 -- 172.21.15.31:0/2864387224 wait complete. 2024-01-19T04:57:00.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-19T04:57:00.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.622+0000 7fb3bddde640 1 Processor -- start 2024-01-19T04:57:00.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.622+0000 7fb3bddde640 1 -- start start 2024-01-19T04:57:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3bddde640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b815f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3bddde640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3b815edd0 0x7fb3b80cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3bddde640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b806c640 con 0x7fb3b80cf830 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3bddde640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b80cfce0 con 0x7fb3b815edd0 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3bddde640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b806c920 con 0x7fb3b813dfc0 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b6ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3b815edd0 0x7fb3b80cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb3b80cf830 0x7fb3b81408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42620/0 (socket says 172.21.15.31:42620) 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b7fff640 1 -- 172.21.15.31:0/4088081745 learned_addr learned my addr 172.21.15.31:0/4088081745 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b77fe640 1 --2- 172.21.15.31:0/4088081745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b6ffd640 1 -- 172.21.15.31:0/4088081745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 msgr2=0x7fb3b815f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b6ffd640 1 --2- 172.21.15.31:0/4088081745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b815f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b6ffd640 1 -- 172.21.15.31:0/4088081745 >> v1:172.21.15.31:6789/0 conn(0x7fb3b80cf830 legacy=0x7fb3b81408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b6ffd640 1 -- 172.21.15.31:0/4088081745 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b806cc00 con 0x7fb3b815edd0 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b67fc640 1 -- 172.21.15.31:0/4088081745 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349688729 0 0) 0x7fb3b806c640 con 0x7fb3b80cf830 2024-01-19T04:57:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.623+0000 7fb3b77fe640 1 --2- 172.21.15.31:0/4088081745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:00.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.624+0000 7fb3b6ffd640 1 --2- 172.21.15.31:0/4088081745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b80cf2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0097a0 tx=0x7fb3ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f0631790351634 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.624+0000 7fb3b67fc640 1 -- 172.21.15.31:0/4088081745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac031830 con 0x7fb3b815edd0 2024-01-19T04:57:00.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.624+0000 7fb3b67fc640 1 -- 172.21.15.31:0/4088081745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac031990 con 0x7fb3b815edd0 2024-01-19T04:57:00.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.624+0000 7fb3b67fc640 1 -- 172.21.15.31:0/4088081745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac031d20 con 0x7fb3b815edd0 2024-01-19T04:57:00.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 -- 172.21.15.31:0/4088081745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 msgr2=0x7fb3b80cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 --2- 172.21.15.31:0/4088081745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b80cf2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0097a0 tx=0x7fb3ac02b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 -- 172.21.15.31:0/4088081745 shutdown_connections 2024-01-19T04:57:00.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 --2- 172.21.15.31:0/4088081745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b80cf2f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 --2- 172.21.15.31:0/4088081745 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 -- 172.21.15.31:0/4088081745 >> 172.21.15.31:0/4088081745 conn(0x7fb3b806aa00 msgr2=0x7fb3b806ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 -- 172.21.15.31:0/4088081745 shutdown_connections 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.625+0000 7fb3bddde640 1 -- 172.21.15.31:0/4088081745 wait complete. 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.626+0000 7fb3bddde640 1 Processor -- start 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.626+0000 7fb3bddde640 1 -- start start 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3bddde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b80cf830 0x7fb3b8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3bddde640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b8139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3bddde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3bddde640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8139ad0 con 0x7fb3b815edd0 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3bddde640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8139d70 con 0x7fb3b80cf830 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3bddde640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b813a050 con 0x7fb3b813dfc0 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b814e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52076/0 (socket says 172.21.15.31:52076) 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 -- 172.21.15.31:0/1768544488 learned_addr learned my addr 172.21.15.31:0/1768544488 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b77fe640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b80cf830 0x7fb3b8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b6ffd640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b8139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 -- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 msgr2=0x7fb3b8139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b8139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 -- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b80cf830 msgr2=0x7fb3b8154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b80cf830 0x7fb3b8154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 -- 172.21.15.31:0/1768544488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac009400 con 0x7fb3b815edd0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b77fe640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b80cf830 0x7fb3b8154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b6ffd640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b8139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.627+0000 7fb3b7fff640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b814e1a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c7a0 tx=0x7fb3b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.628+0000 7fb39ffff640 1 -- 172.21.15.31:0/1768544488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000a640 con 0x7fb3b815edd0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.628+0000 7fb39ffff640 1 -- 172.21.15.31:0/1768544488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b000a7a0 con 0x7fb3b815edd0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.628+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b813a330 con 0x7fb3b815edd0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.628+0000 7fb39ffff640 1 -- 172.21.15.31:0/1768544488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000abd0 con 0x7fb3b815edd0 2024-01-19T04:57:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.628+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b813a8f0 con 0x7fb3b815edd0 2024-01-19T04:57:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.628+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b80cc7c0 con 0x7fb3b815edd0 2024-01-19T04:57:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.629+0000 7fb39ffff640 1 -- 172.21.15.31:0/1768544488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb3b000ad30 con 0x7fb3b815edd0 2024-01-19T04:57:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.629+0000 7fb39ffff640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3a406aed0 0x7fb3a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.629+0000 7fb39ffff640 1 -- 172.21.15.31:0/1768544488 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb3b0088fe0 con 0x7fb3b815edd0 2024-01-19T04:57:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.630+0000 7fb3b77fe640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3a406aed0 0x7fb3a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.630+0000 7fb3b77fe640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3a406aed0 0x7fb3a406d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8006fd0 tx=0x7fb3a8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.634+0000 7fb39ffff640 1 -- 172.21.15.31:0/1768544488 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb3b0053d20 con 0x7fb3b815edd0 2024-01-19T04:57:00.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.848+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb3b813aae0 con 0x7fb3b815edd0 2024-01-19T04:57:00.849 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-19T04:57:00.849+0000 7f70c6e70640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-19T04:57:00.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.850+0000 7fb39ffff640 1 -- 172.21.15.31:0/1768544488 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb3b0018090 con 0x7fb3b815edd0 2024-01-19T04:57:00.851 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3a406aed0 msgr2=0x7fb3a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3a406aed0 0x7fb3a406d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8006fd0 tx=0x7fb3a8008040 comp rx=0 tx=0).stop 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 msgr2=0x7fb3b814e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b814e1a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c7a0 tx=0x7fb3b000cc60 comp rx=0 tx=0).stop 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 shutdown_connections 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb3a406aed0 0x7fb3a406d390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b815edd0 0x7fb3b814e1a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb3b813dfc0 0x7fb3b8139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 --2- 172.21.15.31:0/1768544488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b80cf830 0x7fb3b8154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 >> 172.21.15.31:0/1768544488 conn(0x7fb3b806aa00 msgr2=0x7fb3b813b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:00.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 shutdown_connections 2024-01-19T04:57:00.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.853+0000 7fb3bddde640 1 -- 172.21.15.31:0/1768544488 wait complete. 2024-01-19T04:57:00.855 INFO:tasks.ceph.mds.a.smithi031.stderr: -3248> 2024-01-19T04:56:28.906+0000 7f6a83297640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-19T04:57:00.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-19T04:57:00.883 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-01-19T04:57:00.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b9661640 1 Processor -- start 2024-01-19T04:57:00.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b9661640 1 -- start start 2024-01-19T04:57:00.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b9661640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc9b40cd2e0 0x7fc9b40d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b9661640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b40cce50 con 0x7fc9b40ce090 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b9661640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b40cc240 con 0x7fc9b415b6b0 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b9661640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b40d03b0 con 0x7fc9b40cd2e0 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b27fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc9b40ce090 0x7fc9b40b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42636/0 (socket says 172.21.15.31:42636) 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.969+0000 7fc9b27fc640 1 -- 172.21.15.31:0/2915993984 learned_addr learned my addr 172.21.15.31:0/2915993984 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.970+0000 7fc9b37fe640 1 --2- 172.21.15.31:0/2915993984 >> v2:172.21.15.137:3300/0 conn(0x7fc9b40cd2e0 0x7fc9b40d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.970+0000 7fc9b1ffb640 1 -- 172.21.15.31:0/2915993984 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619377548 0 0) 0x7fc9b40cce50 con 0x7fc9b40ce090 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.970+0000 7fc9b1ffb640 1 -- 172.21.15.31:0/2915993984 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a0003520 con 0x7fc9b40ce090 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.970+0000 7fc9b37fe640 1 -- 172.21.15.31:0/2915993984 >> v1:172.21.15.137:6789/0 conn(0x7fc9b415b6b0 legacy=0x7fc9b40cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.970+0000 7fc9b37fe640 1 -- 172.21.15.31:0/2915993984 >> v1:172.21.15.31:6789/0 conn(0x7fc9b40ce090 legacy=0x7fc9b40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.970+0000 7fc9b37fe640 1 -- 172.21.15.31:0/2915993984 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b40cc520 con 0x7fc9b40cd2e0 2024-01-19T04:57:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.971+0000 7fc9b37fe640 1 --2- 172.21.15.31:0/2915993984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 0x7fc9b40d1c20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009d90 tx=0x7fc9a402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef71701ac32c008d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.971+0000 7fc9b1ffb640 1 -- 172.21.15.31:0/2915993984 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a402cd00 con 0x7fc9b40cd2e0 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.971+0000 7fc9b1ffb640 1 -- 172.21.15.31:0/2915993984 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a402ce60 con 0x7fc9b40cd2e0 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.971+0000 7fc9b1ffb640 1 -- 172.21.15.31:0/2915993984 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4034640 con 0x7fc9b40cd2e0 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.971+0000 7fc9b9661640 1 -- 172.21.15.31:0/2915993984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 msgr2=0x7fc9b40d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.971+0000 7fc9b9661640 1 --2- 172.21.15.31:0/2915993984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 0x7fc9b40d1c20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009d90 tx=0x7fc9a402bf50 comp rx=0 tx=0).stop 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.972+0000 7fc9b9661640 1 -- 172.21.15.31:0/2915993984 shutdown_connections 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.972+0000 7fc9b9661640 1 --2- 172.21.15.31:0/2915993984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 0x7fc9b40d1c20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.972+0000 7fc9b9661640 1 -- 172.21.15.31:0/2915993984 >> 172.21.15.31:0/2915993984 conn(0x7fc9b40d1100 msgr2=0x7fc9b40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.972+0000 7fc9b9661640 1 -- 172.21.15.31:0/2915993984 shutdown_connections 2024-01-19T04:57:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.972+0000 7fc9b9661640 1 -- 172.21.15.31:0/2915993984 wait complete. 2024-01-19T04:57:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.973+0000 7fc9b9661640 1 Processor -- start 2024-01-19T04:57:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.973+0000 7fc9b9661640 1 -- start start 2024-01-19T04:57:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b9661640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 0x7fc9b414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b9661640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc9b40ce090 0x7fc9b40c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b9661640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9b415b6b0 0x7fc9b414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b9661640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40c6b30 con 0x7fc9b415b6b0 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b9661640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b4157c30 con 0x7fc9b40cd2e0 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b9661640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40c5f20 con 0x7fc9b40ce090 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9b415b6b0 0x7fc9b414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9b415b6b0 0x7fc9b414ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52112/0 (socket says 172.21.15.31:52112) 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 -- 172.21.15.31:0/1046769489 learned_addr learned my addr 172.21.15.31:0/1046769489 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b2ffd640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 0x7fc9b414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b27fc640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc9b40ce090 0x7fc9b40c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 -- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc9b40ce090 msgr2=0x7fc9b40c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc9b40ce090 0x7fc9b40c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 -- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 msgr2=0x7fc9b414d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 0x7fc9b414d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 -- 172.21.15.31:0/1046769489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a4009a70 con 0x7fc9b415b6b0 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9b37fe640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9b415b6b0 0x7fc9b414ad50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009d90 tx=0x7fc9a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.974+0000 7fc9ab7fe640 1 -- 172.21.15.31:0/1046769489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4003b90 con 0x7fc9b415b6b0 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.975+0000 7fc9ab7fe640 1 -- 172.21.15.31:0/1046769489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4003cf0 con 0x7fc9b415b6b0 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.975+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b40c6200 con 0x7fc9b415b6b0 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.975+0000 7fc9ab7fe640 1 -- 172.21.15.31:0/1046769489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4004250 con 0x7fc9b415b6b0 2024-01-19T04:57:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.975+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b40c67c0 con 0x7fc9b415b6b0 2024-01-19T04:57:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.975+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b40c6ce0 con 0x7fc9b415b6b0 2024-01-19T04:57:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.976+0000 7fc9ab7fe640 1 -- 172.21.15.31:0/1046769489 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc9a403b040 con 0x7fc9b415b6b0 2024-01-19T04:57:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.976+0000 7fc9ab7fe640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc99406aed0 0x7fc99406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.977+0000 7fc9ab7fe640 1 -- 172.21.15.31:0/1046769489 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6190+0+0 (secure 0 0 0) 0x7fc9a40ab810 con 0x7fc9b415b6b0 2024-01-19T04:57:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.977+0000 7fc9b2ffd640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc99406aed0 0x7fc99406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.977+0000 7fc9b2ffd640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc99406aed0 0x7fc99406d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9b414df70 tx=0x7fc9ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:00.981+0000 7fc9ab7fe640 1 -- 172.21.15.31:0/1046769489 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc9b40c6ce0 con 0x7fc9b415b6b0 2024-01-19T04:57:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.197+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc9b40ccaa0 con 0x7fc9b415b6b0 2024-01-19T04:57:01.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.852+0000 7fc9ab7fe640 1 -- 172.21.15.31:0/1046769489 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9a4076640 con 0x7fc9b415b6b0 2024-01-19T04:57:01.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc99406aed0 msgr2=0x7fc99406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:01.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc99406aed0 0x7fc99406d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9b414df70 tx=0x7fc9ac009040 comp rx=0 tx=0).stop 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9b415b6b0 msgr2=0x7fc9b414ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9b415b6b0 0x7fc9b414ad50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009d90 tx=0x7fc9a4004a40 comp rx=0 tx=0).stop 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 shutdown_connections 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc99406aed0 0x7fc99406d390 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9b415b6b0 0x7fc9b414ad50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc9b40ce090 0x7fc9b40c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 --2- 172.21.15.31:0/1046769489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b40cd2e0 0x7fc9b414d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 >> 172.21.15.31:0/1046769489 conn(0x7fc9b40d1100 msgr2=0x7fc9b415ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 shutdown_connections 2024-01-19T04:57:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:01.855+0000 7fc9b9661640 1 -- 172.21.15.31:0/1046769489 wait complete. 2024-01-19T04:57:01.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-19T04:57:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.022+0000 7f80ca70e640 1 Processor -- start 2024-01-19T04:57:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.022+0000 7f80ca70e640 1 -- start start 2024-01-19T04:57:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.022+0000 7f80ca70e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f80c406c560 0x7f80c4099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.022+0000 7f80ca70e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80c406c940 0x7f80c4099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.022+0000 7f80ca70e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c406bb40 con 0x7f80c40cf430 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.022+0000 7f80ca70e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c40d00a0 con 0x7f80c406c940 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.022+0000 7f80ca70e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c40d0380 con 0x7f80c406c560 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80c406c940 0x7f80c4099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c8cab640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f80c40cf430 0x7f80c40cf880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42638/0 (socket says 172.21.15.31:42638) 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c8cab640 1 -- 172.21.15.31:0/1955543430 learned_addr learned my addr 172.21.15.31:0/1955543430 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c3fff640 1 --2- 172.21.15.31:0/1955543430 >> v2:172.21.15.137:3300/0 conn(0x7f80c406c560 0x7f80c4099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c37fe640 1 -- 172.21.15.31:0/1955543430 >> v2:172.21.15.137:3300/0 conn(0x7f80c406c560 msgr2=0x7f80c4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c37fe640 1 --2- 172.21.15.31:0/1955543430 >> v2:172.21.15.137:3300/0 conn(0x7f80c406c560 0x7f80c4099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c37fe640 1 -- 172.21.15.31:0/1955543430 >> v1:172.21.15.31:6789/0 conn(0x7f80c40cf430 legacy=0x7f80c40cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c37fe640 1 -- 172.21.15.31:0/1955543430 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40d0660 con 0x7f80c406c940 2024-01-19T04:57:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.023+0000 7f80c3fff640 1 --2- 172.21.15.31:0/1955543430 >> v2:172.21.15.137:3300/0 conn(0x7f80c406c560 0x7f80c4099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80c37fe640 1 --2- 172.21.15.31:0/1955543430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c4099f90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f80c4071cd0 tx=0x7f80b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376e0bbc55d19625 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80c2ffd640 1 -- 172.21.15.31:0/1955543430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4031830 con 0x7f80c406c940 2024-01-19T04:57:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80c2ffd640 1 -- 172.21.15.31:0/1955543430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b4031990 con 0x7f80c406c940 2024-01-19T04:57:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80c2ffd640 1 -- 172.21.15.31:0/1955543430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4031d20 con 0x7f80c406c940 2024-01-19T04:57:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 -- 172.21.15.31:0/1955543430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 msgr2=0x7f80c4099f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 --2- 172.21.15.31:0/1955543430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c4099f90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f80c4071cd0 tx=0x7f80b402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 -- 172.21.15.31:0/1955543430 shutdown_connections 2024-01-19T04:57:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 --2- 172.21.15.31:0/1955543430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c4099f90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 --2- 172.21.15.31:0/1955543430 >> v2:172.21.15.137:3300/0 conn(0x7f80c406c560 0x7f80c4099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 -- 172.21.15.31:0/1955543430 >> 172.21.15.31:0/1955543430 conn(0x7f80c40b2bb0 msgr2=0x7f80c40b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 -- 172.21.15.31:0/1955543430 shutdown_connections 2024-01-19T04:57:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.024+0000 7f80ca70e640 1 -- 172.21.15.31:0/1955543430 wait complete. 2024-01-19T04:57:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 Processor -- start 2024-01-19T04:57:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 -- start start 2024-01-19T04:57:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80c406c560 0x7f80c415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c41e6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80c40cf430 0x7f80c40b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41e7360 con 0x7f80c406c940 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41e7640 con 0x7f80c40cf430 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.026+0000 7f80ca70e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41e7920 con 0x7f80c406c560 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c41e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c41e6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52136/0 (socket says 172.21.15.31:52136) 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 -- 172.21.15.31:0/338122536 learned_addr learned my addr 172.21.15.31:0/338122536 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c3fff640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80c406c560 0x7f80c415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c8cab640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80c40cf430 0x7f80c40b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 -- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80c406c560 msgr2=0x7f80c415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80c406c560 0x7f80c415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 -- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80c40cf430 msgr2=0x7f80c40b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80c40cf430 0x7f80c40b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 -- 172.21.15.31:0/338122536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4009400 con 0x7f80c406c940 2024-01-19T04:57:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c37fe640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c41e6910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009890 tx=0x7f80b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c0ff9640 1 -- 172.21.15.31:0/338122536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b403ba70 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c0ff9640 1 -- 172.21.15.31:0/338122536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b403bbf0 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c41e7c00 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80c0ff9640 1 -- 172.21.15.31:0/338122536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4045400 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.027+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c41e8400 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.028+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c41e8120 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.029+0000 7f80c0ff9640 1 -- 172.21.15.31:0/338122536 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f80b404e440 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.029+0000 7f80c0ff9640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a406aed0 0x7f80a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.029+0000 7f80c0ff9640 1 -- 172.21.15.31:0/338122536 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6190+0+0 (secure 0 0 0) 0x7f80b40aba70 con 0x7f80c406c940 2024-01-19T04:57:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.029+0000 7f80c3fff640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a406aed0 0x7f80a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.030+0000 7f80c3fff640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a406aed0 0x7f80a406d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:02.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.033+0000 7f80c0ff9640 1 -- 172.21.15.31:0/338122536 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f80c41e8120 con 0x7f80c406c940 2024-01-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.211+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80c41e8120 con 0x7f80c406c940 2024-01-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.853+0000 7f80c0ff9640 1 -- 172.21.15.31:0/338122536 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80c41e8120 con 0x7f80c406c940 2024-01-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-01-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a406aed0 msgr2=0x7f80a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a406aed0 0x7f80a406d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac008040 comp rx=0 tx=0).stop 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 msgr2=0x7f80c41e6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c41e6910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009890 tx=0x7f80b40046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 shutdown_connections 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f80a406aed0 0x7f80a406d390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80c40cf430 0x7f80c40b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80c406c940 0x7f80c41e6910 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 --2- 172.21.15.31:0/338122536 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80c406c560 0x7f80c415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 >> 172.21.15.31:0/338122536 conn(0x7f80c40b2bb0 msgr2=0x7f80c40b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.857+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 shutdown_connections 2024-01-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.858+0000 7f80ca70e640 1 -- 172.21.15.31:0/338122536 wait complete. 2024-01-19T04:57:02.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.980+0000 7f9a58208640 1 Processor -- start 2024-01-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.980+0000 7f9a58208640 1 -- start start 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a58208640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 0x7f9a50099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a58208640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5015da60 con 0x7f9a50099f90 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a58208640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5006e840 con 0x7f9a500d0720 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a58208640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5009a410 con 0x7f9a5015d680 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a557a3640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9a50099f90 0x7f9a500d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42640/0 (socket says 172.21.15.31:42640) 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a557a3640 1 -- 172.21.15.31:0/2398372049 learned_addr learned my addr 172.21.15.31:0/2398372049 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a55fa4640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 0x7f9a50099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915497720 0 0) 0x7f9a5015da60 con 0x7f9a50099f90 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a3c003520 con 0x7f9a50099f90 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2678786283 0 0) 0x7f9a3c003520 con 0x7f9a50099f90 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 msgr2=0x7f9a50099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 0x7f9a50099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 >> v1:172.21.15.137:6789/0 conn(0x7f9a500d0720 legacy=0x7f9a5006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5006eb20 con 0x7f9a50099f90 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f9a40001670 con 0x7f9a50099f90 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 >> v1:172.21.15.31:6789/0 conn(0x7f9a50099f90 legacy=0x7f9a500d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a3c0039a0 0x7f9a3c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 0x7f9a3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a3c004d00 0x7f9a3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5009a410 con 0x7f9a3c004d00 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c00ba30 con 0x7f9a3c0039a0 2024-01-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c00bbf0 con 0x7f9a5015d680 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a567a5640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a3c004d00 0x7f9a3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.981+0000 7f9a55fa4640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a3c0039a0 0x7f9a3c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.982+0000 7f9a557a3640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 0x7f9a3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.982+0000 7f9a567a5640 1 -- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 msgr2=0x7f9a3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.982+0000 7f9a567a5640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a5015d680 0x7f9a3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.982+0000 7f9a567a5640 1 -- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a3c0039a0 msgr2=0x7f9a3c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.982+0000 7f9a567a5640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a3c0039a0 0x7f9a3c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.982+0000 7f9a567a5640 1 -- 172.21.15.31:0/2398372049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c003520 con 0x7f9a3c004d00 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.982+0000 7f9a567a5640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a3c004d00 0x7f9a3c005180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9a48001e20 tx=0x7f9a3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b06788b3f8ea89c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a48003b70 con 0x7f9a3c004d00 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a48003cd0 con 0x7f9a3c004d00 2024-01-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a54fa2640 1 -- 172.21.15.31:0/2398372049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a48005310 con 0x7f9a3c004d00 2024-01-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a58208640 1 -- 172.21.15.31:0/2398372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a3c004d00 msgr2=0x7f9a3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a58208640 1 --2- 172.21.15.31:0/2398372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a3c004d00 0x7f9a3c005180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9a48001e20 tx=0x7f9a3c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a567a5640 1 -- 172.21.15.31:0/2398372049 reap_dead start 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a58208640 1 -- 172.21.15.31:0/2398372049 shutdown_connections 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a58208640 1 -- 172.21.15.31:0/2398372049 >> 172.21.15.31:0/2398372049 conn(0x7f9a500b2ba0 msgr2=0x7f9a500b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a58208640 1 -- 172.21.15.31:0/2398372049 shutdown_connections 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.983+0000 7f9a58208640 1 -- 172.21.15.31:0/2398372049 wait complete. 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.984+0000 7f9a58208640 1 Processor -- start 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.984+0000 7f9a58208640 1 -- start start 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a501532d0 0x7f9a500cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a500cc400 0x7f9a500ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a50160170 0x7f9a501605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50160b30 con 0x7f9a500cc400 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a501536b0 con 0x7f9a50160170 2024-01-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500cc970 con 0x7f9a501532d0 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a500cc400 0x7f9a500ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a500cc400 0x7f9a500ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52148/0 (socket says 172.21.15.31:52148) 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 -- 172.21.15.31:0/2828228184 learned_addr learned my addr 172.21.15.31:0/2828228184 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a567a5640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a50160170 0x7f9a501605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a55fa4640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a501532d0 0x7f9a500cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 -- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a501532d0 msgr2=0x7f9a500cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a501532d0 0x7f9a500cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 -- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a50160170 msgr2=0x7f9a501605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a50160170 0x7f9a501605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 -- 172.21.15.31:0/2828228184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500ccc50 con 0x7f9a500cc400 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a557a3640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a500cc400 0x7f9a500ca2b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9a40007290 tx=0x7f9a40007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a467fc640 1 -- 172.21.15.31:0/2828228184 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000f940 con 0x7f9a500cc400 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a467fc640 1 -- 172.21.15.31:0/2828228184 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4000fae0 con 0x7f9a500cc400 2024-01-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a500ccf30 con 0x7f9a500cc400 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a467fc640 1 -- 172.21.15.31:0/2828228184 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000fe90 con 0x7f9a500cc400 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.986+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a500cd4f0 con 0x7f9a500cc400 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.987+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a50067b50 con 0x7f9a500cc400 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.987+0000 7f9a467fc640 1 -- 172.21.15.31:0/2828228184 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9a4001d460 con 0x7f9a500cc400 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.988+0000 7f9a467fc640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9a3406af00 0x7f9a3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.988+0000 7f9a467fc640 1 -- 172.21.15.31:0/2828228184 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f9a40089630 con 0x7f9a500cc400 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.988+0000 7f9a55fa4640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9a3406af00 0x7f9a3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.988+0000 7f9a55fa4640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9a3406af00 0x7f9a3406d3c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009b40 tx=0x7f9a4c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:02.993+0000 7f9a467fc640 1 -- 172.21.15.31:0/2828228184 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9a40054560 con 0x7f9a500cc400 2024-01-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.171+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a500cffc0 con 0x7f9a500cc400 2024-01-19T04:57:03.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.855+0000 7f9a467fc640 1 -- 172.21.15.31:0/2828228184 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9a40057b80 con 0x7f9a500cc400 2024-01-19T04:57:03.855 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9a3406af00 msgr2=0x7f9a3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9a3406af00 0x7f9a3406d3c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009b40 tx=0x7f9a4c009190 comp rx=0 tx=0).stop 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a500cc400 msgr2=0x7f9a500ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a500cc400 0x7f9a500ca2b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9a40007290 tx=0x7f9a40007690 comp rx=0 tx=0).stop 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 shutdown_connections 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9a3406af00 0x7f9a3406d3c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a50160170 0x7f9a501605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a500cc400 0x7f9a500ca2b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 --2- 172.21.15.31:0/2828228184 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a501532d0 0x7f9a500cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.859+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 >> 172.21.15.31:0/2828228184 conn(0x7f9a500b2ba0 msgr2=0x7f9a500ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:03.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.860+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 shutdown_connections 2024-01-19T04:57:03.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.860+0000 7f9a58208640 1 -- 172.21.15.31:0/2828228184 wait complete. 2024-01-19T04:57:03.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-01-19T04:57:03.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.985+0000 7f675a485640 1 Processor -- start 2024-01-19T04:57:03.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.985+0000 7f675a485640 1 -- start start 2024-01-19T04:57:03.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f675a485640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 0x7f67540d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f675a485640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f67540b4d20 0x7f67540b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f675a485640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675415d890 con 0x7f67540d1600 2024-01-19T04:57:03.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f675a485640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67540d1a60 con 0x7f67540b4d20 2024-01-19T04:57:03.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f675a485640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67540b56e0 con 0x7f675415d4b0 2024-01-19T04:57:03.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f67537fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67540d1600 0x7f67540b8ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42656/0 (socket says 172.21.15.31:42656) 2024-01-19T04:57:03.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f67537fe640 1 -- 172.21.15.31:0/2914869027 learned_addr learned my addr 172.21.15.31:0/2914869027 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6758a22640 1 --2- 172.21.15.31:0/2914869027 >> v2:172.21.15.137:3300/0 conn(0x7f67540b4d20 0x7f67540b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6753fff640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 0x7f67540d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955073986 0 0) 0x7f675415d890 con 0x7f67540d1600 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6740003520 con 0x7f67540d1600 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162953056 0 0) 0x7f6740003520 con 0x7f67540d1600 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 msgr2=0x7f67540d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 0x7f67540d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 >> v2:172.21.15.137:3300/0 conn(0x7f67540b4d20 msgr2=0x7f67540b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 --2- 172.21.15.31:0/2914869027 >> v2:172.21.15.137:3300/0 conn(0x7f67540b4d20 0x7f67540b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540d1c70 con 0x7f67540d1600 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f6744001670 con 0x7f67540d1600 2024-01-19T04:57:03.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.986+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 >> v1:172.21.15.31:6789/0 conn(0x7f67540d1600 legacy=0x7f67540b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6752ffd640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6740003970 0x7f6740003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6752ffd640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 0x7f67400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6752ffd640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6740004d00 0x7f6740005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540d1a60 con 0x7f6740003970 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540b56e0 con 0x7f6740004d00 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674000b980 con 0x7f675415d4b0 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6753fff640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6740003970 0x7f6740003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6758a22640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6740004d00 0x7f6740005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f67537fe640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 0x7f67400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6753fff640 1 -- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 msgr2=0x7f67400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6753fff640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 0x7f67400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6753fff640 1 -- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6740004d00 msgr2=0x7f6740005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:03.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6753fff640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6740004d00 0x7f6740005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f6753fff640 1 -- 172.21.15.31:0/2914869027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740003520 con 0x7f6740003970 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.987+0000 7f67537fe640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f675415d4b0 0x7f67400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f6753fff640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6740003970 0x7f6740003d90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f673c009ac0 tx=0x7f67400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69354172a9e436c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c02c6b0 con 0x7f6740003970 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c02c810 con 0x7f6740003970 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f6752ffd640 1 -- 172.21.15.31:0/2914869027 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c02cba0 con 0x7f6740003970 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f675a485640 1 -- 172.21.15.31:0/2914869027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6740003970 msgr2=0x7f6740003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f675a485640 1 --2- 172.21.15.31:0/2914869027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6740003970 0x7f6740003d90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f673c009ac0 tx=0x7f67400057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f6758a22640 1 -- 172.21.15.31:0/2914869027 reap_dead start 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f675a485640 1 -- 172.21.15.31:0/2914869027 shutdown_connections 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f675a485640 1 -- 172.21.15.31:0/2914869027 >> 172.21.15.31:0/2914869027 conn(0x7f6754072310 msgr2=0x7f67540cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f675a485640 1 -- 172.21.15.31:0/2914869027 shutdown_connections 2024-01-19T04:57:03.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.988+0000 7f675a485640 1 -- 172.21.15.31:0/2914869027 wait complete. 2024-01-19T04:57:03.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 Processor -- start 2024-01-19T04:57:03.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 -- start start 2024-01-19T04:57:03.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6740003970 0x7f67541507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67540ca370 0x7f67540ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67540c8320 0x7f67540c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540cba00 con 0x7f67540ca370 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c8ce0 con 0x7f67540c8320 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.990+0000 7f675a485640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67540cadf0 con 0x7f6740003970 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67540ca370 0x7f67540ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67540ca370 0x7f67540ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52168/0 (socket says 172.21.15.31:52168) 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 -- 172.21.15.31:0/553654421 learned_addr learned my addr 172.21.15.31:0/553654421 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f6753fff640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6740003970 0x7f67541507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f6758a22640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67540c8320 0x7f67540c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 -- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6740003970 msgr2=0x7f67541507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6740003970 0x7f67541507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 -- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67540c8320 msgr2=0x7f67540c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67540c8320 0x7f67540c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 -- 172.21.15.31:0/553654421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540cb0d0 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f6753fff640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6740003970 0x7f67541507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f6758a22640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67540c8320 0x7f67540c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f67537fe640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67540ca370 0x7f67540ca7f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f674400e7f0 tx=0x7f674400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f6750ff9640 1 -- 172.21.15.31:0/553654421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674400c8d0 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f6750ff9640 1 -- 172.21.15.31:0/553654421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674400ca30 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f6750ff9640 1 -- 172.21.15.31:0/553654421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674400cda0 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67540cb3b0 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.991+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67540cbe90 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.992+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67540696e0 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.992+0000 7f6750ff9640 1 -- 172.21.15.31:0/553654421 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f674401d460 con 0x7f67540ca370 2024-01-19T04:57:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.993+0000 7f6750ff9640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f673406aed0 0x7f673406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:03.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.993+0000 7f6750ff9640 1 -- 172.21.15.31:0/553654421 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5338+0+0 (secure 0 0 0) 0x7f6744089460 con 0x7f67540ca370 2024-01-19T04:57:03.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.993+0000 7f6753fff640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f673406aed0 0x7f673406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:03.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.994+0000 7f6753fff640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f673406aed0 0x7f673406d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f673c036000 tx=0x7f673c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:03.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:03.997+0000 7f6750ff9640 1 -- 172.21.15.31:0/553654421 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6744054530 con 0x7f67540ca370 2024-01-19T04:57:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.177+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67540b47c0 con 0x7f67540ca370 2024-01-19T04:57:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.857+0000 7f6750ff9640 1 -- 172.21.15.31:0/553654421 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v41) v1 ==== 160+0+0 (secure 0 0 0) 0x7f6744057b50 con 0x7f67540ca370 2024-01-19T04:57:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'data_new' removed 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f673406aed0 msgr2=0x7f673406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f673406aed0 0x7f673406d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f673c036000 tx=0x7f673c030000 comp rx=0 tx=0).stop 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67540ca370 msgr2=0x7f67540ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67540ca370 0x7f67540ca7f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f674400e7f0 tx=0x7f674400ecb0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 shutdown_connections 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f673406aed0 0x7f673406d390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67540c8320 0x7f67540c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67540ca370 0x7f67540ca7f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 --2- 172.21.15.31:0/553654421 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6740003970 0x7f67541507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 >> 172.21.15.31:0/553654421 conn(0x7f6754072310 msgr2=0x7f67540b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 shutdown_connections 2024-01-19T04:57:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.861+0000 7f675a485640 1 -- 172.21.15.31:0/553654421 wait complete. 2024-01-19T04:57:04.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-19T04:57:04.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.986+0000 7f52e15b7640 1 Processor -- start 2024-01-19T04:57:04.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.986+0000 7f52e15b7640 1 -- start start 2024-01-19T04:57:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.986+0000 7f52e15b7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f52dc06a5b0 0x7f52dc0cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.986+0000 7f52e15b7640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc072550 con 0x7f52dc06f900 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.986+0000 7f52e15b7640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc15d9d0 con 0x7f52dc0cc1e0 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.986+0000 7f52e15b7640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc06fce0 con 0x7f52dc06a5b0 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.986+0000 7f52daffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52dc06f900 0x7f52dc071e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42668/0 (socket says 172.21.15.31:42668) 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52daffd640 1 -- 172.21.15.31:0/4111559766 learned_addr learned my addr 172.21.15.31:0/4111559766 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974224274 0 0) 0x7f52dc072550 con 0x7f52dc06f900 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52d3fff640 1 --2- 172.21.15.31:0/4111559766 >> v2:172.21.15.137:3300/0 conn(0x7f52dc06a5b0 0x7f52dc0cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c8003520 con 0x7f52dc06f900 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3050367164 0 0) 0x7f52c8003520 con 0x7f52dc06f900 2024-01-19T04:57:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 >> v2:172.21.15.137:3300/0 conn(0x7f52dc06a5b0 msgr2=0x7f52dc0cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 --2- 172.21.15.31:0/4111559766 >> v2:172.21.15.137:3300/0 conn(0x7f52dc06a5b0 0x7f52dc0cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 >> v1:172.21.15.137:6789/0 conn(0x7f52dc0cc1e0 legacy=0x7f52dc0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc06aa40 con 0x7f52dc06f900 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f52cc001670 con 0x7f52dc06f900 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 >> v1:172.21.15.31:6789/0 conn(0x7f52dc06f900 legacy=0x7f52dc071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8003970 0x7f52c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52c80047c0 0x7f52c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c8005180 0x7f52c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc06fce0 con 0x7f52c8005180 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c800c020 con 0x7f52c80047c0 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c800c1e0 con 0x7f52c8003970 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52db7fe640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c8005180 0x7f52c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52daffd640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8003970 0x7f52c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.987+0000 7f52d3fff640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52c80047c0 0x7f52c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.988+0000 7f52db7fe640 1 -- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8003970 msgr2=0x7f52c8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.988+0000 7f52db7fe640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8003970 0x7f52c8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.988+0000 7f52db7fe640 1 -- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52c80047c0 msgr2=0x7f52c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.988+0000 7f52db7fe640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52c80047c0 0x7f52c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.988+0000 7f52db7fe640 1 -- 172.21.15.31:0/4111559766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c8003520 con 0x7f52c8005180 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.988+0000 7f52db7fe640 1 -- 172.21.15.31:0/4111559766 reap_dead start 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.988+0000 7f52db7fe640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c8005180 0x7f52c800b980 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f52d4001e50 tx=0x7f52c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34de6a0718c3521 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4003b70 con 0x7f52c8005180 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4003cf0 con 0x7f52c8005180 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52da7fc640 1 -- 172.21.15.31:0/4111559766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4005310 con 0x7f52c8005180 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52e15b7640 1 -- 172.21.15.31:0/4111559766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c8005180 msgr2=0x7f52c800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52e15b7640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c8005180 0x7f52c800b980 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f52d4001e50 tx=0x7f52c80057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52e15b7640 1 -- 172.21.15.31:0/4111559766 shutdown_connections 2024-01-19T04:57:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52e15b7640 1 --2- 172.21.15.31:0/4111559766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c8005180 0x7f52c800b980 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52e15b7640 1 -- 172.21.15.31:0/4111559766 >> 172.21.15.31:0/4111559766 conn(0x7f52dc0d0000 msgr2=0x7f52dc0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52e15b7640 1 -- 172.21.15.31:0/4111559766 shutdown_connections 2024-01-19T04:57:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.989+0000 7f52e15b7640 1 -- 172.21.15.31:0/4111559766 wait complete. 2024-01-19T04:57:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.990+0000 7f52e15b7640 1 Processor -- start 2024-01-19T04:57:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.990+0000 7f52e15b7640 1 -- start start 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52e15b7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8005180 0x7f52dc1f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52e15b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1f2dd0 0x7f52dc09e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52e15b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52dc0b3d20 0x7f52dc15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52e15b7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0b2bf0 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52e15b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc1f32d0 con 0x7f52dc0b3d20 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52e15b7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc1f35b0 con 0x7f52c8005180 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1f2dd0 0x7f52dc09e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1f2dd0 0x7f52dc09e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52184/0 (socket says 172.21.15.31:52184) 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 -- 172.21.15.31:0/860940975 learned_addr learned my addr 172.21.15.31:0/860940975 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52daffd640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8005180 0x7f52dc1f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52db7fe640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52dc0b3d20 0x7f52dc15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 -- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8005180 msgr2=0x7f52dc1f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8005180 0x7f52dc1f01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 -- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52dc0b3d20 msgr2=0x7f52dc15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52dc0b3d20 0x7f52dc15ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52d3fff640 1 -- 172.21.15.31:0/860940975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc1f3890 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.991+0000 7f52db7fe640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52dc0b3d20 0x7f52dc15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.992+0000 7f52d3fff640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1f2dd0 0x7f52dc09e320 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f52c40118e0 tx=0x7f52c4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.992+0000 7f52d37fe640 1 -- 172.21.15.31:0/860940975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400faf0 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.992+0000 7f52d37fe640 1 -- 172.21.15.31:0/860940975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400fc50 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.992+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc1f3b70 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.992+0000 7f52d37fe640 1 -- 172.21.15.31:0/860940975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4014440 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.992+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc09e860 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.993+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc074900 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.993+0000 7f52d37fe640 1 -- 172.21.15.31:0/860940975 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f52c401d400 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.994+0000 7f52d37fe640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52bc06af00 0x7f52bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.994+0000 7f52d37fe640 1 -- 172.21.15.31:0/860940975 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f52c4089bc0 con 0x7f52dc1f2dd0 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.994+0000 7f52daffd640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52bc06af00 0x7f52bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.994+0000 7f52daffd640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52bc06af00 0x7f52bc06d3c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f52cc001050 tx=0x7f52cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:04.998+0000 7f52d37fe640 1 -- 172.21.15.31:0/860940975 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f52c4054d90 con 0x7f52dc1f2dd0 2024-01-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.176+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f52dc1f3d00 con 0x7f52dc1f2dd0 2024-01-19T04:57:05.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.179+0000 7f52d37fe640 1 -- 172.21.15.31:0/860940975 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f52c40583b0 con 0x7f52dc1f2dd0 2024-01-19T04:57:05.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52bc06af00 msgr2=0x7f52bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52bc06af00 0x7f52bc06d3c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f52cc001050 tx=0x7f52cc009040 comp rx=0 tx=0).stop 2024-01-19T04:57:05.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1f2dd0 msgr2=0x7f52dc09e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1f2dd0 0x7f52dc09e320 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f52c40118e0 tx=0x7f52c4011da0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 shutdown_connections 2024-01-19T04:57:05.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52bc06af00 0x7f52bc06d3c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52dc0b3d20 0x7f52dc15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1f2dd0 0x7f52dc09e320 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 --2- 172.21.15.31:0/860940975 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f52c8005180 0x7f52dc1f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.181+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 >> 172.21.15.31:0/860940975 conn(0x7f52dc0d0000 msgr2=0x7f52dc0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.182+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 shutdown_connections 2024-01-19T04:57:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.182+0000 7f52e15b7640 1 -- 172.21.15.31:0/860940975 wait complete. 2024-01-19T04:57:05.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-01-19T04:57:05.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.294+0000 7f1da4b61640 1 Processor -- start 2024-01-19T04:57:05.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.294+0000 7f1da4b61640 1 -- start start 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1da4b61640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1da00ce760 0x7f1da01449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1da4b61640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da015f3e0 con 0x7f1da015ee40 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1da4b61640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da015f6c0 con 0x7f1da00ceb40 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1da4b61640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da00cde30 con 0x7f1da00ce760 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1d9e59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1da00ce760 0x7f1da01449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1d9ed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1da015ee40 0x7f1da00cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42678/0 (socket says 172.21.15.31:42678) 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/3955127242 learned_addr learned my addr 172.21.15.31:0/3955127242 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1d9e59c640 1 -- 172.21.15.31:0/3955127242 >> v1:172.21.15.137:6789/0 conn(0x7f1da00ceb40 legacy=0x7f1da00cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1d9d59a640 1 -- 172.21.15.31:0/3955127242 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707902008 0 0) 0x7f1da015f3e0 con 0x7f1da015ee40 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1d9e59c640 1 -- 172.21.15.31:0/3955127242 >> v1:172.21.15.31:6789/0 conn(0x7f1da015ee40 legacy=0x7f1da00cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.295+0000 7f1d9e59c640 1 -- 172.21.15.31:0/3955127242 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da015f9a0 con 0x7f1da00ce760 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.296+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/3955127242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da00ce760 0x7f1da01449d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1d980097a0 tx=0x7f1d9802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813f2b352f60ec70 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.296+0000 7f1d9d59a640 1 -- 172.21.15.31:0/3955127242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98030610 con 0x7f1da00ce760 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.296+0000 7f1d9d59a640 1 -- 172.21.15.31:0/3955127242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d98030770 con 0x7f1da00ce760 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.296+0000 7f1d9d59a640 1 -- 172.21.15.31:0/3955127242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98030b00 con 0x7f1da00ce760 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.297+0000 7f1da4b61640 1 -- 172.21.15.31:0/3955127242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da00ce760 msgr2=0x7f1da01449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.297+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3955127242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da00ce760 0x7f1da01449d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1d980097a0 tx=0x7f1d9802b960 comp rx=0 tx=0).stop 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.297+0000 7f1da4b61640 1 -- 172.21.15.31:0/3955127242 shutdown_connections 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.297+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3955127242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da00ce760 0x7f1da01449d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.297+0000 7f1da4b61640 1 -- 172.21.15.31:0/3955127242 >> 172.21.15.31:0/3955127242 conn(0x7f1da006a790 msgr2=0x7f1da006aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.297+0000 7f1da4b61640 1 -- 172.21.15.31:0/3955127242 shutdown_connections 2024-01-19T04:57:05.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.297+0000 7f1da4b61640 1 -- 172.21.15.31:0/3955127242 wait complete. 2024-01-19T04:57:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.298+0000 7f1da4b61640 1 Processor -- start 2024-01-19T04:57:05.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.298+0000 7f1da4b61640 1 -- start start 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1da4b61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da00ce760 0x7f1da015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1da4b61640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da015ee40 0x7f1da015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1da4b61640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1da015ca40 0x7f1da01e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1da4b61640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da01f35e0 con 0x7f1da015ee40 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1da4b61640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da01e5110 con 0x7f1da00ce760 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1da4b61640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da015d510 con 0x7f1da015ca40 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1d9dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da015ee40 0x7f1da015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1d9dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da015ee40 0x7f1da015c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52192/0 (socket says 172.21.15.31:52192) 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1d9dd9b640 1 -- 172.21.15.31:0/3703391169 learned_addr learned my addr 172.21.15.31:0/3703391169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da00ce760 0x7f1da015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.299+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1da015ca40 0x7f1da01e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d9dd9b640 1 -- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1da015ca40 msgr2=0x7f1da01e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d9dd9b640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1da015ca40 0x7f1da01e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d9dd9b640 1 -- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da00ce760 msgr2=0x7f1da015ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d9dd9b640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da00ce760 0x7f1da015ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d9dd9b640 1 -- 172.21.15.31:0/3703391169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d98009400 con 0x7f1da015ee40 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d9dd9b640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da015ee40 0x7f1da015c500 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c450 tx=0x7f1d8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d96ffd640 1 -- 172.21.15.31:0/3703391169 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880052d0 con 0x7f1da015ee40 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d96ffd640 1 -- 172.21.15.31:0/3703391169 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88005430 con 0x7f1da015ee40 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da015cf50 con 0x7f1da015ee40 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1d96ffd640 1 -- 172.21.15.31:0/3703391169 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88012870 con 0x7f1da015ee40 2024-01-19T04:57:05.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.300+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da01e4da0 con 0x7f1da015ee40 2024-01-19T04:57:05.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.301+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da01e52a0 con 0x7f1da015ee40 2024-01-19T04:57:05.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.301+0000 7f1d96ffd640 1 -- 172.21.15.31:0/3703391169 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1d880129d0 con 0x7f1da015ee40 2024-01-19T04:57:05.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.302+0000 7f1d96ffd640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1d8006aed0 0x7f1d8006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.302+0000 7f1d96ffd640 1 -- 172.21.15.31:0/3703391169 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1d88088d50 con 0x7f1da015ee40 2024-01-19T04:57:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.302+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1d8006aed0 0x7f1d8006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.302+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1d8006aed0 0x7f1d8006d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d98036000 tx=0x7f1d98008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:05.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.306+0000 7f1d96ffd640 1 -- 172.21.15.31:0/3703391169 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1d8808c030 con 0x7f1da015ee40 2024-01-19T04:57:05.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.555+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f1da00b2490 con 0x7f1da015ee40 2024-01-19T04:57:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.866+0000 7f1d96ffd640 1 -- 172.21.15.31:0/3703391169 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v116) v1 ==== 147+0+0 (secure 0 0 0) 0x7f1d88053fb0 con 0x7f1da015ee40 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1d8006aed0 msgr2=0x7f1d8006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1d8006aed0 0x7f1d8006d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d98036000 tx=0x7f1d98008030 comp rx=0 tx=0).stop 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da015ee40 msgr2=0x7f1da015c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da015ee40 0x7f1da015c500 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c450 tx=0x7f1d8800c910 comp rx=0 tx=0).stop 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 shutdown_connections 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f1d8006aed0 0x7f1d8006d390 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1da015ca40 0x7f1da01e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da015ee40 0x7f1da015c500 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 --2- 172.21.15.31:0/3703391169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da00ce760 0x7f1da015ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 >> 172.21.15.31:0/3703391169 conn(0x7f1da006a790 msgr2=0x7f1da00cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 shutdown_connections 2024-01-19T04:57:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.868+0000 7f1da4b61640 1 -- 172.21.15.31:0/3703391169 wait complete. 2024-01-19T04:57:05.879 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2024-01-19T04:57:05.879 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-19T04:57:05.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:05.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.983+0000 7f34c8615640 1 Processor -- start 2024-01-19T04:57:05.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.983+0000 7f34c8615640 1 -- start start 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c8615640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f34c015de70 0x7f34c015e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c8615640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34c00b2d40 0x7f34c00b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c8615640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c00cb8b0 con 0x7f34c01580f0 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c8615640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c006c300 con 0x7f34c00b2d40 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c8615640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c015f740 con 0x7f34c015de70 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c6bb2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34c00b2d40 0x7f34c00b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c63b1640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f34c01580f0 0x7f34c015f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42688/0 (socket says 172.21.15.31:42688) 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c63b1640 1 -- 172.21.15.31:0/1665894645 learned_addr learned my addr 172.21.15.31:0/1665894645 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c5bb0640 1 --2- 172.21.15.31:0/1665894645 >> v2:172.21.15.137:3300/0 conn(0x7f34c015de70 0x7f34c015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c6bb2640 1 -- 172.21.15.31:0/1665894645 >> v2:172.21.15.137:3300/0 conn(0x7f34c015de70 msgr2=0x7f34c015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c6bb2640 1 --2- 172.21.15.31:0/1665894645 >> v2:172.21.15.137:3300/0 conn(0x7f34c015de70 0x7f34c015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c6bb2640 1 -- 172.21.15.31:0/1665894645 >> v1:172.21.15.31:6789/0 conn(0x7f34c01580f0 legacy=0x7f34c015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c6bb2640 1 -- 172.21.15.31:0/1665894645 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00b36b0 con 0x7f34c00b2d40 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c53af640 1 -- 172.21.15.31:0/1665894645 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676615328 0 0) 0x7f34c00cb8b0 con 0x7f34c01580f0 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.984+0000 7f34c5bb0640 1 --2- 172.21.15.31:0/1665894645 >> v2:172.21.15.137:3300/0 conn(0x7f34c015de70 0x7f34c015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.985+0000 7f34c6bb2640 1 --2- 172.21.15.31:0/1665894645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c00b2d40 0x7f34c00b3170 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f34c00b3d00 tx=0x7f34b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f5a26b69205e04 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.985+0000 7f34c53af640 1 -- 172.21.15.31:0/1665894645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b802cc80 con 0x7f34c00b2d40 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.985+0000 7f34c53af640 1 -- 172.21.15.31:0/1665894645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b802cde0 con 0x7f34c00b2d40 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.985+0000 7f34c53af640 1 -- 172.21.15.31:0/1665894645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8034660 con 0x7f34c00b2d40 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 -- 172.21.15.31:0/1665894645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c00b2d40 msgr2=0x7f34c00b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 --2- 172.21.15.31:0/1665894645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c00b2d40 0x7f34c00b3170 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f34c00b3d00 tx=0x7f34b802bed0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 -- 172.21.15.31:0/1665894645 shutdown_connections 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 --2- 172.21.15.31:0/1665894645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c00b2d40 0x7f34c00b3170 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 --2- 172.21.15.31:0/1665894645 >> v2:172.21.15.137:3300/0 conn(0x7f34c015de70 0x7f34c015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 -- 172.21.15.31:0/1665894645 >> 172.21.15.31:0/1665894645 conn(0x7f34c006abe0 msgr2=0x7f34c0066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 -- 172.21.15.31:0/1665894645 shutdown_connections 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.986+0000 7f34c8615640 1 -- 172.21.15.31:0/1665894645 wait complete. 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.987+0000 7f34c8615640 1 Processor -- start 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.987+0000 7f34c8615640 1 -- start start 2024-01-19T04:57:05.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c8615640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c00b2d40 0x7f34c00c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c8615640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c01580f0 0x7f34c00b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c8615640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34c015de70 0x7f34c00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c8615640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0148190 con 0x7f34c01580f0 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c8615640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00c9860 con 0x7f34c00b2d40 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c8615640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0147580 con 0x7f34c015de70 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c01580f0 0x7f34c00b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c01580f0 0x7f34c00b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52206/0 (socket says 172.21.15.31:52206) 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 -- 172.21.15.31:0/3627031691 learned_addr learned my addr 172.21.15.31:0/3627031691 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c63b1640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c00b2d40 0x7f34c00c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c6bb2640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34c015de70 0x7f34c00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 -- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34c015de70 msgr2=0x7f34c00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34c015de70 0x7f34c00b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 -- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c00b2d40 msgr2=0x7f34c00c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:05.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c00b2d40 0x7f34c00c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 -- 172.21.15.31:0/3627031691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8009a70 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c5bb0640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c01580f0 0x7f34c00b5a00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c7f0 tx=0x7f34b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34b6ffd640 1 -- 172.21.15.31:0/3627031691 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0007820 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34b6ffd640 1 -- 172.21.15.31:0/3627031691 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b0007980 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c0147860 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.988+0000 7f34b6ffd640 1 -- 172.21.15.31:0/3627031691 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0007cf0 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.989+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c0147e20 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.990+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c015ec20 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.990+0000 7f34b6ffd640 1 -- 172.21.15.31:0/3627031691 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f34b00113f0 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.990+0000 7f34b6ffd640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f34a406aed0 0x7f34a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.990+0000 7f34b6ffd640 1 -- 172.21.15.31:0/3627031691 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f34b0089680 con 0x7f34c01580f0 2024-01-19T04:57:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.991+0000 7f34c63b1640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f34a406aed0 0x7f34a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.991+0000 7f34c63b1640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f34a406aed0 0x7f34a406d390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f34bc000f60 tx=0x7f34bc007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:05.994+0000 7f34b6ffd640 1 -- 172.21.15.31:0/3627031691 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f34c015ec20 con 0x7f34c01580f0 2024-01-19T04:57:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.172+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34c015ec20 con 0x7f34c01580f0 2024-01-19T04:57:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.173+0000 7f34b6ffd640 1 -- 172.21.15.31:0/3627031691 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12949 (secure 0 0 0) 0x7f34c015ec20 con 0x7f34c01580f0 2024-01-19T04:57:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:06.172 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":41,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:04.856727+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/968278604":"2024-01-20T04:57:00.850929+0000","172.21.15.31:6834/968278604":"2024-01-20T04:57:00.850929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f34a406aed0 msgr2=0x7f34a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f34a406aed0 0x7f34a406d390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f34bc000f60 tx=0x7f34bc007b50 comp rx=0 tx=0).stop 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c01580f0 msgr2=0x7f34c00b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c01580f0 0x7f34c00b5a00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c7f0 tx=0x7f34b000ccb0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 shutdown_connections 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f34a406aed0 0x7f34a406d390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34c015de70 0x7f34c00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c01580f0 0x7f34c00b5a00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 --2- 172.21.15.31:0/3627031691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c00b2d40 0x7f34c00c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 >> 172.21.15.31:0/3627031691 conn(0x7f34c006abe0 msgr2=0x7f34c00b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 shutdown_connections 2024-01-19T04:57:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.175+0000 7f34c8615640 1 -- 172.21.15.31:0/3627031691 wait complete. 2024-01-19T04:57:06.186 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:06.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:06.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.290+0000 7f5f96b9e640 1 Processor -- start 2024-01-19T04:57:06.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.290+0000 7f5f96b9e640 1 -- start start 2024-01-19T04:57:06.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f96b9e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f900d3190 0x7f5f900b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f96b9e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5f900d19e0 0x7f5f900d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f96b9e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f900b36a0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f96b9e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900b4320 con 0x7f5f900d3190 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f96b9e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900b3980 con 0x7f5f900d19e0 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f95b9c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f900d3190 0x7f5f900b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f9539b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5f9015d4f0 0x7f5f900d3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42704/0 (socket says 172.21.15.31:42704) 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.291+0000 7f5f9539b640 1 -- 172.21.15.31:0/3391834934 learned_addr learned my addr 172.21.15.31:0/3391834934 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f9639d640 1 --2- 172.21.15.31:0/3391834934 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d19e0 0x7f5f900d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f95b9c640 1 -- 172.21.15.31:0/3391834934 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d19e0 msgr2=0x7f5f900d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f95b9c640 1 --2- 172.21.15.31:0/3391834934 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d19e0 0x7f5f900d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f95b9c640 1 -- 172.21.15.31:0/3391834934 >> v1:172.21.15.31:6789/0 conn(0x7f5f9015d4f0 legacy=0x7f5f900d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f95b9c640 1 -- 172.21.15.31:0/3391834934 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900b3c60 con 0x7f5f900d3190 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f94b9a640 1 -- 172.21.15.31:0/3391834934 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559284125 0 0) 0x7f5f900b36a0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f9639d640 1 --2- 172.21.15.31:0/3391834934 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d19e0 0x7f5f900d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.292+0000 7f5f95b9c640 1 --2- 172.21.15.31:0/3391834934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3190 0x7f5f900b4740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f900d0e40 tx=0x7f5f8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7b006576c47aeb server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f94b9a640 1 -- 172.21.15.31:0/3391834934 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88031830 con 0x7f5f900d3190 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f94b9a640 1 -- 172.21.15.31:0/3391834934 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f88031990 con 0x7f5f900d3190 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f94b9a640 1 -- 172.21.15.31:0/3391834934 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88031d20 con 0x7f5f900d3190 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 -- 172.21.15.31:0/3391834934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3190 msgr2=0x7f5f900b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/3391834934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3190 0x7f5f900b4740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f900d0e40 tx=0x7f5f8802b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 -- 172.21.15.31:0/3391834934 shutdown_connections 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/3391834934 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d19e0 0x7f5f900d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/3391834934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3190 0x7f5f900b4740 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 -- 172.21.15.31:0/3391834934 >> 172.21.15.31:0/3391834934 conn(0x7f5f9008cdb0 msgr2=0x7f5f9009c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 -- 172.21.15.31:0/3391834934 shutdown_connections 2024-01-19T04:57:06.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.293+0000 7f5f96b9e640 1 -- 172.21.15.31:0/3391834934 wait complete. 2024-01-19T04:57:06.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 Processor -- start 2024-01-19T04:57:06.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 -- start start 2024-01-19T04:57:06.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d19e0 0x7f5f900b8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f900d3190 0x7f5f90151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9015d4f0 0x7f5f900cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900b85d0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900b6ff0 con 0x7f5f900d19e0 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.295+0000 7f5f96b9e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900b72d0 con 0x7f5f900d3190 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9015d4f0 0x7f5f900cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9015d4f0 0x7f5f900cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52228/0 (socket says 172.21.15.31:52228) 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 -- 172.21.15.31:0/2250034820 learned_addr learned my addr 172.21.15.31:0/2250034820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9539b640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f900d3190 0x7f5f90151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f95b9c640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d19e0 0x7f5f900b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 -- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f900d3190 msgr2=0x7f5f90151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f900d3190 0x7f5f90151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 -- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d19e0 msgr2=0x7f5f900b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d19e0 0x7f5f900b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 -- 172.21.15.31:0/2250034820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880093a0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f9639d640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9015d4f0 0x7f5f900cc7f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000cef0 tx=0x7f5f8000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2250034820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000d840 con 0x7f5f9015d4f0 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2250034820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8000d9c0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f900b75b0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2250034820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000dd70 con 0x7f5f9015d4f0 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.296+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f900b7b70 con 0x7f5f9015d4f0 2024-01-19T04:57:06.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.297+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f900b7d90 con 0x7f5f9015d4f0 2024-01-19T04:57:06.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.298+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2250034820 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5f8001e3b0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.298+0000 7f5f8e7fc640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.298+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2250034820 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5f80089e20 con 0x7f5f9015d4f0 2024-01-19T04:57:06.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.298+0000 7f5f95b9c640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.299+0000 7f5f95b9c640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5f88036000 tx=0x7f5f88008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.302+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2250034820 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5f800550b0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.479+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f900d4370 con 0x7f5f9015d4f0 2024-01-19T04:57:06.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.479+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2250034820 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f8001a0a0 con 0x7f5f9015d4f0 2024-01-19T04:57:06.479 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 msgr2=0x7f5f7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5f88036000 tx=0x7f5f88008030 comp rx=0 tx=0).stop 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9015d4f0 msgr2=0x7f5f900cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9015d4f0 0x7f5f900cc7f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000cef0 tx=0x7f5f8000a710 comp rx=0 tx=0).stop 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 shutdown_connections 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9015d4f0 0x7f5f900cc7f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f900d3190 0x7f5f90151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 --2- 172.21.15.31:0/2250034820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d19e0 0x7f5f900b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 >> 172.21.15.31:0/2250034820 conn(0x7f5f9008cdb0 msgr2=0x7f5f900b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 shutdown_connections 2024-01-19T04:57:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.481+0000 7f5f96b9e640 1 -- 172.21.15.31:0/2250034820 wait complete. 2024-01-19T04:57:06.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-01-19T04:57:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.593+0000 7fa14e8a6640 1 Processor -- start 2024-01-19T04:57:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.593+0000 7fa14e8a6640 1 -- start start 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14e8a6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa14815d300 0x7fa14806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14e8a6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa148071df0 0x7fa1480aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14e8a6640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1480b3510 con 0x7fa1480b3030 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14e8a6640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1480b37f0 con 0x7fa148071df0 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14e8a6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480b3ad0 con 0x7fa14815d300 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14ce43640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa148071df0 0x7fa1480aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14ce43640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa148071df0 0x7fa1480aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52240/0 (socket says 172.21.15.31:52240) 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14ce43640 1 -- 172.21.15.31:0/3566473571 learned_addr learned my addr 172.21.15.31:0/3566473571 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa147fff640 1 --2- 172.21.15.31:0/3566473571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa14815d300 0x7fa14806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14ce43640 1 -- 172.21.15.31:0/3566473571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa14815d300 msgr2=0x7fa14806ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14ce43640 1 --2- 172.21.15.31:0/3566473571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa14815d300 0x7fa14806ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14ce43640 1 -- 172.21.15.31:0/3566473571 >> v1:172.21.15.137:6789/0 conn(0x7fa1480b3030 legacy=0x7fa148160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa14ce43640 1 -- 172.21.15.31:0/3566473571 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14806a650 con 0x7fa148071df0 2024-01-19T04:57:06.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.594+0000 7fa147fff640 1 --2- 172.21.15.31:0/3566473571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa14815d300 0x7fa14806ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.595+0000 7fa14ce43640 1 --2- 172.21.15.31:0/3566473571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148071df0 0x7fa1480aeea0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa140009e50 tx=0x7fa14002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb03512aa1988a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.595+0000 7fa146ffd640 1 -- 172.21.15.31:0/3566473571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14002cb00 con 0x7fa148071df0 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.595+0000 7fa146ffd640 1 -- 172.21.15.31:0/3566473571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14002cc60 con 0x7fa148071df0 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.595+0000 7fa146ffd640 1 -- 172.21.15.31:0/3566473571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1400343f0 con 0x7fa148071df0 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 -- 172.21.15.31:0/3566473571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148071df0 msgr2=0x7fa1480aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/3566473571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148071df0 0x7fa1480aeea0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa140009e50 tx=0x7fa14002bf60 comp rx=0 tx=0).stop 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 -- 172.21.15.31:0/3566473571 shutdown_connections 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/3566473571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148071df0 0x7fa1480aeea0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/3566473571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa14815d300 0x7fa14806ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 -- 172.21.15.31:0/3566473571 >> 172.21.15.31:0/3566473571 conn(0x7fa14806d750 msgr2=0x7fa14806db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:06.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 -- 172.21.15.31:0/3566473571 shutdown_connections 2024-01-19T04:57:06.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.596+0000 7fa14e8a6640 1 -- 172.21.15.31:0/3566473571 wait complete. 2024-01-19T04:57:06.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.597+0000 7fa14e8a6640 1 Processor -- start 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.597+0000 7fa14e8a6640 1 -- start start 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14e8a6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa148071df0 0x7fa1480b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14e8a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1480b3030 0x7fa1480b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14e8a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14815d300 0x7fa1480b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14e8a6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1481603e0 con 0x7fa14815d300 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14e8a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14814f4e0 con 0x7fa1480b3030 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14e8a6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480b0dc0 con 0x7fa148071df0 2024-01-19T04:57:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14815d300 0x7fa1480b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14815d300 0x7fa1480b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52250/0 (socket says 172.21.15.31:52250) 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 -- 172.21.15.31:0/2966632442 learned_addr learned my addr 172.21.15.31:0/2966632442 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa1477fe640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1480b3030 0x7fa1480b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa147fff640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa148071df0 0x7fa1480b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 -- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa148071df0 msgr2=0x7fa1480b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa148071df0 0x7fa1480b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 -- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1480b3030 msgr2=0x7fa1480b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1480b3030 0x7fa1480b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 -- 172.21.15.31:0/2966632442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140009b30 con 0x7fa14815d300 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa147fff640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa148071df0 0x7fa1480b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa14ce43640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14815d300 0x7fa1480b20f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa140036040 tx=0x7fa140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.598+0000 7fa144ff9640 1 -- 172.21.15.31:0/2966632442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140009510 con 0x7fa14815d300 2024-01-19T04:57:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.599+0000 7fa144ff9640 1 -- 172.21.15.31:0/2966632442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14003e450 con 0x7fa14815d300 2024-01-19T04:57:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.599+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14815fe20 con 0x7fa14815d300 2024-01-19T04:57:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.599+0000 7fa144ff9640 1 -- 172.21.15.31:0/2966632442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14003e800 con 0x7fa14815d300 2024-01-19T04:57:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.599+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1480b13b0 con 0x7fa14815d300 2024-01-19T04:57:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.600+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1480696e0 con 0x7fa14815d300 2024-01-19T04:57:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.600+0000 7fa144ff9640 1 -- 172.21.15.31:0/2966632442 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa14003e960 con 0x7fa14815d300 2024-01-19T04:57:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.600+0000 7fa144ff9640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa12806aed0 0x7fa12806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.601+0000 7fa144ff9640 1 -- 172.21.15.31:0/2966632442 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa1400ac010 con 0x7fa14815d300 2024-01-19T04:57:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.601+0000 7fa147fff640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa12806aed0 0x7fa12806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.601+0000 7fa147fff640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa12806aed0 0x7fa12806d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa138006fd0 tx=0x7fa138008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.605+0000 7fa144ff9640 1 -- 172.21.15.31:0/2966632442 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa140077270 con 0x7fa14815d300 2024-01-19T04:57:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.860+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fa148071bd0 con 0x7fa14815d300 2024-01-19T04:57:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.867+0000 7fa144ff9640 1 -- 172.21.15.31:0/2966632442 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v117) v1 ==== 137+0+0 (secure 0 0 0) 0x7fa14007a890 con 0x7fa14815d300 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa12806aed0 msgr2=0x7fa12806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa12806aed0 0x7fa12806d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa138006fd0 tx=0x7fa138008040 comp rx=0 tx=0).stop 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14815d300 msgr2=0x7fa1480b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14815d300 0x7fa1480b20f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa140036040 tx=0x7fa140004a40 comp rx=0 tx=0).stop 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 shutdown_connections 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa12806aed0 0x7fa12806d390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14815d300 0x7fa1480b20f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1480b3030 0x7fa1480b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 --2- 172.21.15.31:0/2966632442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa148071df0 0x7fa1480b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 >> 172.21.15.31:0/2966632442 conn(0x7fa14806d750 msgr2=0x7fa1481527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 shutdown_connections 2024-01-19T04:57:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.869+0000 7fa14e8a6640 1 -- 172.21.15.31:0/2966632442 wait complete. 2024-01-19T04:57:06.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-19T04:57:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.984+0000 7f9a0cc9b640 1 Processor -- start 2024-01-19T04:57:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.984+0000 7f9a0cc9b640 1 -- start start 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0cc9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0815b8c0 0x7f9a0815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0cc9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a0813e8e0 0x7f9a0813ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0cc9b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9a08071a00 0x7f9a08071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0cc9b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0815fd50 con 0x7f9a0815b8c0 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0cc9b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080723d0 con 0x7f9a08071a00 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0cc9b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0815f280 con 0x7f9a0813e8e0 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0815b8c0 0x7f9a0815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0815b8c0 0x7f9a0815ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52254/0 (socket says 172.21.15.31:52254) 2024-01-19T04:57:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 -- 172.21.15.31:0/1568281069 learned_addr learned my addr 172.21.15.31:0/1568281069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a06d9d640 1 --2- 172.21.15.31:0/1568281069 >> v2:172.21.15.137:3300/0 conn(0x7f9a08071a00 0x7f9a08071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 -- 172.21.15.31:0/1568281069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a0813e8e0 msgr2=0x7f9a0813ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a05d9b640 1 --2- 172.21.15.31:0/1568281069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a0813e8e0 0x7f9a0813ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 --2- 172.21.15.31:0/1568281069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a0813e8e0 0x7f9a0813ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 -- 172.21.15.31:0/1568281069 >> v2:172.21.15.137:3300/0 conn(0x7f9a08071a00 msgr2=0x7f9a08071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 --2- 172.21.15.31:0/1568281069 >> v2:172.21.15.137:3300/0 conn(0x7f9a08071a00 0x7f9a08071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a0659c640 1 -- 172.21.15.31:0/1568281069 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a08071570 con 0x7f9a0815b8c0 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a06d9d640 1 --2- 172.21.15.31:0/1568281069 >> v2:172.21.15.137:3300/0 conn(0x7f9a08071a00 0x7f9a08071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.985+0000 7f9a05d9b640 1 --2- 172.21.15.31:0/1568281069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a0813e8e0 0x7f9a0813ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.986+0000 7f9a0659c640 1 --2- 172.21.15.31:0/1568281069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0815b8c0 0x7f9a0815ed40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a0815ec70 tx=0x7f99f802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786218e3ff19a1a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.986+0000 7f9a0559a640 1 -- 172.21.15.31:0/1568281069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f802ca70 con 0x7f9a0815b8c0 2024-01-19T04:57:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.986+0000 7f9a0559a640 1 -- 172.21.15.31:0/1568281069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f802cbd0 con 0x7f9a0815b8c0 2024-01-19T04:57:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.986+0000 7f9a0559a640 1 -- 172.21.15.31:0/1568281069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f80343f0 con 0x7f9a0815b8c0 2024-01-19T04:57:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.986+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/1568281069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0815b8c0 msgr2=0x7f9a0815ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.986+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/1568281069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0815b8c0 0x7f9a0815ed40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a0815ec70 tx=0x7f99f802bed0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.987+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/1568281069 shutdown_connections 2024-01-19T04:57:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.987+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/1568281069 >> v2:172.21.15.137:3300/0 conn(0x7f9a08071a00 0x7f9a08071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.987+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/1568281069 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a0813e8e0 0x7f9a0813ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.987+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/1568281069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0815b8c0 0x7f9a0815ed40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.987+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/1568281069 >> 172.21.15.31:0/1568281069 conn(0x7f9a080b1c50 msgr2=0x7f9a080b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.987+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/1568281069 shutdown_connections 2024-01-19T04:57:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.987+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/1568281069 wait complete. 2024-01-19T04:57:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.988+0000 7f9a0cc9b640 1 Processor -- start 2024-01-19T04:57:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.988+0000 7f9a0cc9b640 1 -- start start 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0cc9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a08071a00 0x7f9a080b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0cc9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0813e8e0 0x7f9a0806e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0cc9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0815b8c0 0x7f9a0813d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0cc9b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080b3dc0 con 0x7f9a0813e8e0 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0cc9b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080b3b20 con 0x7f9a0815b8c0 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0cc9b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0814ccd0 con 0x7f9a08071a00 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0813e8e0 0x7f9a0806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0813e8e0 0x7f9a0806e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52268/0 (socket says 172.21.15.31:52268) 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 -- 172.21.15.31:0/311125036 learned_addr learned my addr 172.21.15.31:0/311125036 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a06d9d640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0815b8c0 0x7f9a0813d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0659c640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a08071a00 0x7f9a080b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 -- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a08071a00 msgr2=0x7f9a080b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a08071a00 0x7f9a080b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 -- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0815b8c0 msgr2=0x7f9a0813d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0815b8c0 0x7f9a0813d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 -- 172.21.15.31:0/311125036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8009b30 con 0x7f9a0813e8e0 2024-01-19T04:57:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a06d9d640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0815b8c0 0x7f9a0813d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a0659c640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a08071a00 0x7f9a080b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f9a05d9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0813e8e0 0x7f9a0806e970 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c7f0 tx=0x7f99fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.989+0000 7f99f6ffd640 1 -- 172.21.15.31:0/311125036 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc007a30 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.990+0000 7f99f6ffd640 1 -- 172.21.15.31:0/311125036 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc007b90 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.990+0000 7f99f6ffd640 1 -- 172.21.15.31:0/311125036 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00ce40 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.990+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0814cfb0 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.990+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0814d570 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.991+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a08074900 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.991+0000 7f99f6ffd640 1 -- 172.21.15.31:0/311125036 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f99fc0113f0 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.991+0000 7f99f6ffd640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.991+0000 7f99f6ffd640 1 -- 172.21.15.31:0/311125036 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f99fc0893b0 con 0x7f9a0813e8e0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.992+0000 7f9a0659c640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.992+0000 7f9a0659c640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99f8035000 tx=0x7f99f8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:06.995+0000 7f99f6ffd640 1 -- 172.21.15.31:0/311125036 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f99fc054610 con 0x7f9a0813e8e0 2024-01-19T04:57:07.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.174+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9a080727b0 con 0x7f9a0813e8e0 2024-01-19T04:57:07.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.177+0000 7f99f6ffd640 1 -- 172.21.15.31:0/311125036 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99fc057c30 con 0x7f9a0813e8e0 2024-01-19T04:57:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 msgr2=0x7f99e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99f8035000 tx=0x7f99f8030000 comp rx=0 tx=0).stop 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0813e8e0 msgr2=0x7f9a0806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0813e8e0 0x7f9a0806e970 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c7f0 tx=0x7f99fc00cbf0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 shutdown_connections 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f99e806aed0 0x7f99e806d390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0815b8c0 0x7f9a0813d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0813e8e0 0x7f9a0806e970 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 --2- 172.21.15.31:0/311125036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a08071a00 0x7f9a080b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 >> 172.21.15.31:0/311125036 conn(0x7f9a080b1c50 msgr2=0x7f9a080c5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 shutdown_connections 2024-01-19T04:57:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.179+0000 7f9a0cc9b640 1 -- 172.21.15.31:0/311125036 wait complete. 2024-01-19T04:57:07.189 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-19T04:57:07.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:57:07.228 DEBUG:teuthology.orchestra.run.smithi137:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:57:07.276 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-19T04:57:07.276 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:07.276 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:07.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057151,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20760 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249472k,nr_inodes=812368,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-19T04:57:07.291 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:57:07.291 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:07.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:57:07.310 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:07.310 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:07.349 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057151,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20760 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249472k,nr_inodes=812368,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-19T04:57:07.350 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:57:07.350 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:07.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:57:07.368 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-19T04:57:07.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:57:07.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.491+0000 7f11a653c640 1 Processor -- start 2024-01-19T04:57:07.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.491+0000 7f11a653c640 1 -- start start 2024-01-19T04:57:07.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f11a653c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f11a00d2e50 0x7f11a00b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f11a653c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a0160df0 con 0x7f11a0160a10 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f11a653c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a00d1db0 con 0x7f11a00b4d00 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f11a653c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a00d2090 con 0x7f11a00d2e50 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f119ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f11a0160a10 0x7f11a00d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42716/0 (socket says 172.21.15.31:42716) 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f119ffff640 1 -- 172.21.15.31:0/1919740376 learned_addr learned my addr 172.21.15.31:0/1919740376 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f119f7fe640 1 --2- 172.21.15.31:0/1919740376 >> v2:172.21.15.137:3300/0 conn(0x7f11a00d2e50 0x7f11a00b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.492+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888433561 0 0) 0x7f11a0160df0 con 0x7f11a0160a10 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1188003520 con 0x7f11a0160a10 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1005260013 0 0) 0x7f1188003520 con 0x7f11a0160a10 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 >> v2:172.21.15.137:3300/0 conn(0x7f11a00d2e50 msgr2=0x7f11a00b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 --2- 172.21.15.31:0/1919740376 >> v2:172.21.15.137:3300/0 conn(0x7f11a00d2e50 0x7f11a00b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 >> v1:172.21.15.137:6789/0 conn(0x7f11a00b4d00 legacy=0x7f11a00b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00d2370 con 0x7f11a0160a10 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f1194001670 con 0x7f11a0160a10 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 >> v1:172.21.15.31:6789/0 conn(0x7f11a0160a10 legacy=0x7f11a00d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1188003970 0x7f1188003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11880047c0 0x7f1188004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1188005180 0x7f118800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00d2090 con 0x7f1188003970 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118800c020 con 0x7f11880047c0 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118800c1e0 con 0x7f1188005180 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119ffff640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1188003970 0x7f1188003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119f7fe640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11880047c0 0x7f1188004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f11a4ad9640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1188005180 0x7f118800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119ffff640 1 -- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1188005180 msgr2=0x7f118800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119ffff640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1188005180 0x7f118800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119ffff640 1 -- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11880047c0 msgr2=0x7f1188004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119ffff640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11880047c0 0x7f1188004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f119ffff640 1 -- 172.21.15.31:0/1919740376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188003520 con 0x7f1188003970 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.493+0000 7f11a4ad9640 1 -- 172.21.15.31:0/1919740376 reap_dead start 2024-01-19T04:57:07.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.494+0000 7f119ffff640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1188003970 0x7f1188003d90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f119400c240 tx=0x7f11880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2e236acd787809 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.494+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194005ad0 con 0x7f1188003970 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.494+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194005c30 con 0x7f1188003970 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.494+0000 7f119effd640 1 -- 172.21.15.31:0/1919740376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119402caf0 con 0x7f1188003970 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.495+0000 7f11a653c640 1 -- 172.21.15.31:0/1919740376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1188003970 msgr2=0x7f1188003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.495+0000 7f11a653c640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1188003970 0x7f1188003d90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f119400c240 tx=0x7f11880057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.495+0000 7f11a653c640 1 -- 172.21.15.31:0/1919740376 shutdown_connections 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.495+0000 7f11a653c640 1 --2- 172.21.15.31:0/1919740376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1188003970 0x7f1188003d90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.495+0000 7f11a653c640 1 -- 172.21.15.31:0/1919740376 >> 172.21.15.31:0/1919740376 conn(0x7f11a0072310 msgr2=0x7f11a00cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.495+0000 7f11a653c640 1 -- 172.21.15.31:0/1919740376 shutdown_connections 2024-01-19T04:57:07.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.495+0000 7f11a653c640 1 -- 172.21.15.31:0/1919740376 wait complete. 2024-01-19T04:57:07.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.496+0000 7f11a653c640 1 Processor -- start 2024-01-19T04:57:07.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.496+0000 7f11a653c640 1 -- start start 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a653c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1188003970 0x7f11a00ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a653c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11a0154740 0x7f11a014b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a653c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0156c90 0x7f11a0157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a653c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a014bab0 con 0x7f11a0156c90 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a653c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0155bf0 con 0x7f1188003970 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a653c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0155ed0 con 0x7f11a0154740 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0156c90 0x7f11a0157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0156c90 0x7f11a0157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52284/0 (socket says 172.21.15.31:52284) 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 -- 172.21.15.31:0/2601413779 learned_addr learned my addr 172.21.15.31:0/2601413779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f119f7fe640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11a0154740 0x7f11a014b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f119ffff640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1188003970 0x7f11a00ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 -- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11a0154740 msgr2=0x7f11a014b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11a0154740 0x7f11a014b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 -- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1188003970 msgr2=0x7f11a00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1188003970 0x7f11a00ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 -- 172.21.15.31:0/2601413779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a01561b0 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f119ffff640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1188003970 0x7f11a00ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.497+0000 7f11a4ad9640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0156c90 0x7f11a0157ae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1198007fd0 tx=0x7f119800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.498+0000 7f119cff9640 1 -- 172.21.15.31:0/2601413779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198003b80 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.498+0000 7f119cff9640 1 -- 172.21.15.31:0/2601413779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198003d00 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.498+0000 7f119cff9640 1 -- 172.21.15.31:0/2601413779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198005310 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.498+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a0156490 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.498+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a0155880 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.499+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a01569b0 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.499+0000 7f119cff9640 1 -- 172.21.15.31:0/2601413779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1198005470 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.499+0000 7f119cff9640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f118c06af00 0x7f118c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.500+0000 7f119cff9640 1 -- 172.21.15.31:0/2601413779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1198088aa0 con 0x7f11a0156c90 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.500+0000 7f119ffff640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f118c06af00 0x7f118c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.500+0000 7f119ffff640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f118c06af00 0x7f118c06d3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1194034000 tx=0x7f1194009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:07.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.504+0000 7f119cff9640 1 -- 172.21.15.31:0/2601413779 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f11a01569b0 con 0x7f11a0156c90 2024-01-19T04:57:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.722+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11a00696e0 con 0x7f11a0156c90 2024-01-19T04:57:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.722+0000 7f119cff9640 1 -- 172.21.15.31:0/2601413779 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3701 (secure 0 0 0) 0x7f1198019070 con 0x7f11a0156c90 2024-01-19T04:57:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:07.722 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/4228897961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4228897961},{"type":"v1","addr":"172.21.15.137:6836","nonce":4228897961}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4829,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/452521575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":452521575},{"type":"v1","addr":"172.21.15.137:6837","nonce":452521575}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1900734781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1900734781},{"type":"v1","addr":"172.21.15.31:6833","nonce":1900734781}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5829,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3430985894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3430985894},{"type":"v1","addr":"172.21.15.31:6835","nonce":3430985894}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[]} 2024-01-19T04:57:07.722 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f118c06af00 msgr2=0x7f118c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f118c06af00 0x7f118c06d3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1194034000 tx=0x7f1194009030 comp rx=0 tx=0).stop 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0156c90 msgr2=0x7f11a0157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0156c90 0x7f11a0157ae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1198007fd0 tx=0x7f119800ead0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 shutdown_connections 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f118c06af00 0x7f118c06d3c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0156c90 0x7f11a0157ae0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11a0154740 0x7f11a014b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 --2- 172.21.15.31:0/2601413779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1188003970 0x7f11a00ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 >> 172.21.15.31:0/2601413779 conn(0x7f11a0072310 msgr2=0x7f11a00cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 shutdown_connections 2024-01-19T04:57:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.724+0000 7f11a653c640 1 -- 172.21.15.31:0/2601413779 wait complete. 2024-01-19T04:57:07.735 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-19T04:57:07.735 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-19T04:57:07.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-19T04:57:07.735 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:57:07.735 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-19T04:57:07.735 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-19T04:57:07.735 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-19T04:57:07.735 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:57:07.736 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-19T04:57:07.736 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-19T04:57:07.736 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-19T04:57:07.736 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:57:07.736 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-19T04:57:07.736 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-19T04:57:07.736 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-19T04:57:07.737 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:57:07.744 INFO:tasks.ceph.mds.a:Stopped 2024-01-19T04:57:07.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-19T04:57:07.745 INFO:tasks.ceph.mds.c:Stopped 2024-01-19T04:57:07.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-19T04:57:07.748 INFO:tasks.ceph.mds.b:Stopped 2024-01-19T04:57:07.748 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-19T04:57:07.749 INFO:tasks.ceph.mds.d:Stopped 2024-01-19T04:57:07.749 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-19T04:57:07.786 INFO:tasks.ceph.mds.a:Started 2024-01-19T04:57:07.786 DEBUG:teuthology.parallel:result is None 2024-01-19T04:57:07.787 INFO:tasks.ceph.mds.c:Started 2024-01-19T04:57:07.787 DEBUG:teuthology.parallel:result is None 2024-01-19T04:57:07.791 INFO:tasks.ceph.mds.b:Started 2024-01-19T04:57:07.791 DEBUG:teuthology.parallel:result is None 2024-01-19T04:57:07.791 INFO:tasks.ceph.mds.d:Started 2024-01-19T04:57:07.791 DEBUG:teuthology.parallel:result is None 2024-01-19T04:57:07.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-19T04:57:07.862 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2024-01-19T04:57:07.862 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-01-19T04:57:07.872 INFO:tasks.ceph.mds.b.smithi137.stdout:starting mds.b at 2024-01-19T04:57:07.872 INFO:tasks.ceph.mds.d.smithi137.stdout:starting mds.d at 2024-01-19T04:57:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.899+0000 7f7011eec640 1 Processor -- start 2024-01-19T04:57:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.899+0000 7f7011eec640 1 -- start start 2024-01-19T04:57:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f7011eec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c15d570 0x7f700c06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f7011eec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f700c0d1820 0x7f700c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f7011eec640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0b0450 con 0x7f700c0cbed0 2024-01-19T04:57:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f7011eec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0cb760 con 0x7f700c0d1820 2024-01-19T04:57:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f7011eec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0cba40 con 0x7f700c15d570 2024-01-19T04:57:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f700affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f700c0d1820 0x7f700c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f700affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f700c0d1820 0x7f700c06c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52344/0 (socket says 172.21.15.31:52344) 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f700affd640 1 -- 172.21.15.31:0/1859812804 learned_addr learned my addr 172.21.15.31:0/1859812804 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.900+0000 7f700b7fe640 1 --2- 172.21.15.31:0/1859812804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c15d570 0x7f700c06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.901+0000 7f700affd640 1 -- 172.21.15.31:0/1859812804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c15d570 msgr2=0x7f700c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.901+0000 7f700affd640 1 --2- 172.21.15.31:0/1859812804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c15d570 0x7f700c06bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.901+0000 7f700affd640 1 -- 172.21.15.31:0/1859812804 >> v1:172.21.15.137:6789/0 conn(0x7f700c0cbed0 legacy=0x7f700c0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.901+0000 7f700affd640 1 -- 172.21.15.31:0/1859812804 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0ce630 con 0x7f700c0d1820 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.901+0000 7f700affd640 1 --2- 172.21.15.31:0/1859812804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1820 0x7f700c06c510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7000009870 tx=0x7f700002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56eab34e399a5ff2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.901+0000 7f700a7fc640 1 -- 172.21.15.31:0/1859812804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000031640 con 0x7f700c0d1820 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f700a7fc640 1 -- 172.21.15.31:0/1859812804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000317a0 con 0x7f700c0d1820 2024-01-19T04:57:07.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f700a7fc640 1 -- 172.21.15.31:0/1859812804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000031b30 con 0x7f700c0d1820 2024-01-19T04:57:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 -- 172.21.15.31:0/1859812804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1820 msgr2=0x7f700c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 --2- 172.21.15.31:0/1859812804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1820 0x7f700c06c510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7000009870 tx=0x7f700002b8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 -- 172.21.15.31:0/1859812804 shutdown_connections 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 --2- 172.21.15.31:0/1859812804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c0d1820 0x7f700c06c510 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 --2- 172.21.15.31:0/1859812804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c15d570 0x7f700c06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 -- 172.21.15.31:0/1859812804 >> 172.21.15.31:0/1859812804 conn(0x7f700c08afe0 msgr2=0x7f700c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 -- 172.21.15.31:0/1859812804 shutdown_connections 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.902+0000 7f7011eec640 1 -- 172.21.15.31:0/1859812804 wait complete. 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.903+0000 7f7011eec640 1 Processor -- start 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.903+0000 7f7011eec640 1 -- start start 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f7011eec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c0cbed0 0x7f700c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f7011eec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1820 0x7f700c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f7011eec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c15d570 0x7f700c150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f7011eec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c151680 con 0x7f700c15d570 2024-01-19T04:57:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f7011eec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c152050 con 0x7f700c0d1820 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f7011eec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c150a70 con 0x7f700c0cbed0 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f700bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c15d570 0x7f700c150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f700bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c15d570 0x7f700c150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52350/0 (socket says 172.21.15.31:52350) 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.904+0000 7f700bfff640 1 -- 172.21.15.31:0/2346251372 learned_addr learned my addr 172.21.15.31:0/2346251372 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700b7fe640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c0cbed0 0x7f700c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700affd640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1820 0x7f700c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700bfff640 1 -- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c0cbed0 msgr2=0x7f700c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700bfff640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c0cbed0 0x7f700c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700bfff640 1 -- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1820 msgr2=0x7f700c151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700bfff640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1820 0x7f700c151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700bfff640 1 -- 172.21.15.31:0/2346251372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000094d0 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f700bfff640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c15d570 0x7f700c150220 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f700400c6b0 tx=0x7f700400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f6ff3fff640 1 -- 172.21.15.31:0/2346251372 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004003b70 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f6ff3fff640 1 -- 172.21.15.31:0/2346251372 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7004003cf0 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c150d50 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f6ff3fff640 1 -- 172.21.15.31:0/2346251372 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004003ea0 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.905+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c151310 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.906+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c1517f0 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.906+0000 7f6ff3fff640 1 -- 172.21.15.31:0/2346251372 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7004005310 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.907+0000 7f6ff3fff640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff806aed0 0x7f6ff806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.907+0000 7f6ff3fff640 1 -- 172.21.15.31:0/2346251372 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7004089090 con 0x7f700c15d570 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.907+0000 7f700b7fe640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff806aed0 0x7f6ff806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.908+0000 7f700b7fe640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff806aed0 0x7f6ff806d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc006fd0 tx=0x7f6ffc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:07.911+0000 7f6ff3fff640 1 -- 172.21.15.31:0/2346251372 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7004054270 con 0x7f700c15d570 2024-01-19T04:57:08.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:08.100+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f700c074900 con 0x7f700c15d570 2024-01-19T04:57:09.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.192+0000 7f6ff3fff640 1 -- 172.21.15.31:0/2346251372 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7004057890 con 0x7f700c15d570 2024-01-19T04:57:09.192 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2024-01-19T04:57:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff806aed0 msgr2=0x7f6ff806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff806aed0 0x7f6ff806d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc006fd0 tx=0x7f6ffc008040 comp rx=0 tx=0).stop 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c15d570 msgr2=0x7f700c150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c15d570 0x7f700c150220 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f700400c6b0 tx=0x7f700400cb70 comp rx=0 tx=0).stop 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 shutdown_connections 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ff806aed0 0x7f6ff806d390 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700c15d570 0x7f700c150220 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c0d1820 0x7f700c151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 --2- 172.21.15.31:0/2346251372 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f700c0cbed0 0x7f700c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 >> 172.21.15.31:0/2346251372 conn(0x7f700c08afe0 msgr2=0x7f700c06e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 shutdown_connections 2024-01-19T04:57:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.195+0000 7f7011eec640 1 -- 172.21.15.31:0/2346251372 wait complete. 2024-01-19T04:57:09.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-19T04:57:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.316+0000 7f623abd7640 1 Processor -- start 2024-01-19T04:57:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.316+0000 7f623abd7640 1 -- start start 2024-01-19T04:57:09.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.316+0000 7f623abd7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f62340ce550 0x7f62340b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.316+0000 7f623abd7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62340b2f90 0x7f62340cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.316+0000 7f623abd7640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6234070e50 con 0x7f6234150a50 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.316+0000 7f623abd7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340cba30 con 0x7f62340b2f90 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.316+0000 7f623abd7640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340b33a0 con 0x7f62340ce550 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6239174640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62340b2f90 0x7f62340cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6238973640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6234150a50 0x7f6234159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42750/0 (socket says 172.21.15.31:42750) 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6238973640 1 -- 172.21.15.31:0/1739295972 learned_addr learned my addr 172.21.15.31:0/1739295972 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6233fff640 1 --2- 172.21.15.31:0/1739295972 >> v2:172.21.15.137:3300/0 conn(0x7f62340ce550 0x7f62340b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6239174640 1 -- 172.21.15.31:0/1739295972 >> v2:172.21.15.137:3300/0 conn(0x7f62340ce550 msgr2=0x7f62340b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6239174640 1 --2- 172.21.15.31:0/1739295972 >> v2:172.21.15.137:3300/0 conn(0x7f62340ce550 0x7f62340b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6239174640 1 -- 172.21.15.31:0/1739295972 >> v1:172.21.15.31:6789/0 conn(0x7f6234150a50 legacy=0x7f6234159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f6239174640 1 -- 172.21.15.31:0/1739295972 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234150e30 con 0x7f62340b2f90 2024-01-19T04:57:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.317+0000 7f62337fe640 1 -- 172.21.15.31:0/1739295972 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698313777 0 0) 0x7f6234070e50 con 0x7f6234150a50 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f6239174640 1 --2- 172.21.15.31:0/1739295972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62340b2f90 0x7f62340cf170 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f623406e580 tx=0x7f621c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5244c987fb04344c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f62337fe640 1 -- 172.21.15.31:0/1739295972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c031830 con 0x7f62340b2f90 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f62337fe640 1 -- 172.21.15.31:0/1739295972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c031990 con 0x7f62340b2f90 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f62337fe640 1 -- 172.21.15.31:0/1739295972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c031d20 con 0x7f62340b2f90 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f623abd7640 1 -- 172.21.15.31:0/1739295972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62340b2f90 msgr2=0x7f62340cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f623abd7640 1 --2- 172.21.15.31:0/1739295972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62340b2f90 0x7f62340cf170 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f623406e580 tx=0x7f621c02b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f623abd7640 1 -- 172.21.15.31:0/1739295972 shutdown_connections 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f623abd7640 1 --2- 172.21.15.31:0/1739295972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62340b2f90 0x7f62340cf170 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f623abd7640 1 --2- 172.21.15.31:0/1739295972 >> v2:172.21.15.137:3300/0 conn(0x7f62340ce550 0x7f62340b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.318+0000 7f623abd7640 1 -- 172.21.15.31:0/1739295972 >> 172.21.15.31:0/1739295972 conn(0x7f623406a790 msgr2=0x7f623406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.319+0000 7f623abd7640 1 -- 172.21.15.31:0/1739295972 shutdown_connections 2024-01-19T04:57:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.319+0000 7f623abd7640 1 -- 172.21.15.31:0/1739295972 wait complete. 2024-01-19T04:57:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.320+0000 7f623abd7640 1 Processor -- start 2024-01-19T04:57:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.320+0000 7f623abd7640 1 -- start start 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62340b2f90 0x7f62341f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62340ce550 0x7f6234154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234150a50 0x7f6234153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234155a60 con 0x7f6234150a50 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234155d40 con 0x7f62340ce550 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6234156020 con 0x7f62340b2f90 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234150a50 0x7f6234153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234150a50 0x7f6234153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52362/0 (socket says 172.21.15.31:52362) 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 -- 172.21.15.31:0/1727377547 learned_addr learned my addr 172.21.15.31:0/1727377547 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6238973640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62340b2f90 0x7f62341f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6233fff640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62340ce550 0x7f6234154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 -- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62340b2f90 msgr2=0x7f62341f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62340b2f90 0x7f62341f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 -- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62340ce550 msgr2=0x7f6234154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62340ce550 0x7f6234154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 -- 172.21.15.31:0/1727377547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009400 con 0x7f6234150a50 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6233fff640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62340ce550 0x7f6234154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f6239174640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234150a50 0x7f6234153200 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f621c034040 tx=0x7f621c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f62317fa640 1 -- 172.21.15.31:0/1727377547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0317d0 con 0x7f6234150a50 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f62317fa640 1 -- 172.21.15.31:0/1727377547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c03a9e0 con 0x7f6234150a50 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6234156300 con 0x7f6234150a50 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f62317fa640 1 -- 172.21.15.31:0/1727377547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c03ad90 con 0x7f6234150a50 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.321+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62341f4000 con 0x7f6234150a50 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.323+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234156820 con 0x7f6234150a50 2024-01-19T04:57:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.323+0000 7f62317fa640 1 -- 172.21.15.31:0/1727377547 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f621c0443f0 con 0x7f6234150a50 2024-01-19T04:57:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.323+0000 7f62317fa640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f621406af00 0x7f621406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.323+0000 7f62317fa640 1 -- 172.21.15.31:0/1727377547 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f621c0ab480 con 0x7f6234150a50 2024-01-19T04:57:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.323+0000 7f6238973640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f621406af00 0x7f621406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.324+0000 7f6238973640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f621406af00 0x7f621406d3c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f622c0012e0 tx=0x7f622c007ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.327+0000 7f62317fa640 1 -- 172.21.15.31:0/1727377547 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6234156820 con 0x7f6234150a50 2024-01-19T04:57:09.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.576+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6234156820 con 0x7f6234150a50 2024-01-19T04:57:09.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.577+0000 7f62317fa640 1 -- 172.21.15.31:0/1727377547 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f6234156820 con 0x7f6234150a50 2024-01-19T04:57:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:09.576 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl9vwvARAAZduX/PeYlqDH8VDbvZiELg==", 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl6TSLDRAAGB2c1AX6e5HAuMDl6tCbKw==", 2024-01-19T04:57:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplMbeJCBAAkHOlhUk5I9r4TWFRo9L/eA==", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplQl/gFBAA3VbnYQWE/2FHLv7opt1f9w==", 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB7AKplbqKsNRAAikErjUtmLKvlkeH7GGL1UQ==", 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB+AKplVZk5NBAA6FIuSbDo0CCaftcB9zQbmw==", 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCBAKplP8CiMhAABJdd6kUn7DlZFCikXiggGQ==", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCEAKplPAbxMBAANYNlBOGgzR60cDNfMpZTDA==", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCKAKplzlV2NxAAsprGSRQCPR98cdSNEBqmqA==", 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCNAKplqnZRNhAA2gX5iijb/OorAr9M0kxMBw==", 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCQAKplG4yxNBAAIgP3l7C0BuCHY3ZzK0ezuQ==", 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCTAKplKnckMhAApf2mP+F7NZ39VBHDkNVBTg==", 2024-01-19T04:57:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw==", 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs_data" 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl11pPIhAAr+yfn0112ldu1NcYFzRIZg==", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKpl9grnHhAAnUsK7YtYaC8ZPOX4W84PbA==", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-01-19T04:57:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB4AKplog8YKBAA9SYGPnH5B4/9JInpxCxziw==", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3AKplRU9kIBAA1HYggMF7NTcKjXTzd1tRqg==", 2024-01-19T04:57:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3AKplwl9qORAAdjRFheF+sxCmZe31mpCjVw==", 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3AKpl6LqONRAAoCHvKhaqK1R2nkcFvmGb1A==", 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-01-19T04:57:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:09.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f621406af00 msgr2=0x7f621406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f621406af00 0x7f621406d3c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f622c0012e0 tx=0x7f622c007ed0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234150a50 msgr2=0x7f6234153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234150a50 0x7f6234153200 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f621c034040 tx=0x7f621c0046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 shutdown_connections 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f621406af00 0x7f621406d3c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234150a50 0x7f6234153200 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62340ce550 0x7f6234154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 --2- 172.21.15.31:0/1727377547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62340b2f90 0x7f62341f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 >> 172.21.15.31:0/1727377547 conn(0x7f623406a790 msgr2=0x7f623406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 shutdown_connections 2024-01-19T04:57:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.579+0000 7f623abd7640 1 -- 172.21.15.31:0/1727377547 wait complete. 2024-01-19T04:57:09.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.692+0000 7f05ced1f640 1 Processor -- start 2024-01-19T04:57:09.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.692+0000 7f05ced1f640 1 -- start start 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05ced1f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05c815eb70 0x7f05c80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05ced1f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c806bf30 con 0x7f05c80cca00 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05ced1f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c80cb9e0 con 0x7f05c815d8a0 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05ced1f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c80cbcc0 con 0x7f05c815eb70 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05c7fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05c815eb70 0x7f05c80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05cd2bc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05c80cca00 0x7f05c815f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60632/0 (socket says 172.21.15.31:60632) 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05cd2bc640 1 -- 172.21.15.31:0/3797230828 learned_addr learned my addr 172.21.15.31:0/3797230828 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05c7fff640 1 -- 172.21.15.31:0/3797230828 >> v1:172.21.15.137:6789/0 conn(0x7f05c815d8a0 legacy=0x7f05c806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05c7fff640 1 -- 172.21.15.31:0/3797230828 >> v1:172.21.15.31:6789/0 conn(0x7f05c80cca00 legacy=0x7f05c815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.693+0000 7f05c7fff640 1 -- 172.21.15.31:0/3797230828 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c80cbfa0 con 0x7f05c815eb70 2024-01-19T04:57:09.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.694+0000 7f05c7fff640 1 --2- 172.21.15.31:0/3797230828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80cc4c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0097a0 tx=0x7f05bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79cc3968b69faa8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.694+0000 7f05c77fe640 1 -- 172.21.15.31:0/3797230828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc02e070 con 0x7f05c815eb70 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.694+0000 7f05c77fe640 1 -- 172.21.15.31:0/3797230828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc0315a0 con 0x7f05c815eb70 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.694+0000 7f05c77fe640 1 -- 172.21.15.31:0/3797230828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc031930 con 0x7f05c815eb70 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.695+0000 7f05ced1f640 1 -- 172.21.15.31:0/3797230828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 msgr2=0x7f05c80cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.695+0000 7f05ced1f640 1 --2- 172.21.15.31:0/3797230828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80cc4c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0097a0 tx=0x7f05bc02b840 comp rx=0 tx=0).stop 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.695+0000 7f05ced1f640 1 -- 172.21.15.31:0/3797230828 shutdown_connections 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.695+0000 7f05ced1f640 1 --2- 172.21.15.31:0/3797230828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80cc4c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.695+0000 7f05ced1f640 1 -- 172.21.15.31:0/3797230828 >> 172.21.15.31:0/3797230828 conn(0x7f05c806abe0 msgr2=0x7f05c8066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.695+0000 7f05ced1f640 1 -- 172.21.15.31:0/3797230828 shutdown_connections 2024-01-19T04:57:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.695+0000 7f05ced1f640 1 -- 172.21.15.31:0/3797230828 wait complete. 2024-01-19T04:57:09.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.696+0000 7f05ced1f640 1 Processor -- start 2024-01-19T04:57:09.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.696+0000 7f05ced1f640 1 -- start start 2024-01-19T04:57:09.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05c80cca00 0x7f05c81556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05c815d8a0 0x7f05c80b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c80b4ab0 con 0x7f05c815eb70 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c80c5af0 con 0x7f05c80cca00 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c80b6d10 con 0x7f05c815d8a0 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80b63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47590/0 (socket says 172.21.15.31:47590) 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 -- 172.21.15.31:0/2637886287 learned_addr learned my addr 172.21.15.31:0/2637886287 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 -- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05c815d8a0 msgr2=0x7f05c80b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05c815d8a0 0x7f05c80b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 -- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05c80cca00 msgr2=0x7f05c81556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05c80cca00 0x7f05c81556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 -- 172.21.15.31:0/2637886287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009400 con 0x7f05c815eb70 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05cd2bc640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80b63b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f05c000ca20 tx=0x7f05c000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:09.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05c57fa640 1 -- 172.21.15.31:0/2637886287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c000abd0 con 0x7f05c815eb70 2024-01-19T04:57:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05c57fa640 1 -- 172.21.15.31:0/2637886287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c000ad30 con 0x7f05c815eb70 2024-01-19T04:57:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c80b6a30 con 0x7f05c815eb70 2024-01-19T04:57:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05c57fa640 1 -- 172.21.15.31:0/2637886287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0012620 con 0x7f05c815eb70 2024-01-19T04:57:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.697+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c80c5780 con 0x7f05c815eb70 2024-01-19T04:57:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.698+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c806b180 con 0x7f05c815eb70 2024-01-19T04:57:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.699+0000 7f05c57fa640 1 -- 172.21.15.31:0/2637886287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f05c0007070 con 0x7f05c815eb70 2024-01-19T04:57:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.699+0000 7f05c57fa640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f05b406aed0 0x7f05b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.699+0000 7f05c57fa640 1 -- 172.21.15.31:0/2637886287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05c0089100 con 0x7f05c815eb70 2024-01-19T04:57:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.700+0000 7f05ccabb640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f05b406aed0 0x7f05b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.700+0000 7f05ccabb640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f05b406aed0 0x7f05b406d390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f05b8001560 tx=0x7f05b8007990 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:09.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.704+0000 7f05c57fa640 1 -- 172.21.15.31:0/2637886287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f05c00543b0 con 0x7f05c815eb70 2024-01-19T04:57:09.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.887+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05c80696e0 con 0x7f05c815eb70 2024-01-19T04:57:09.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.887+0000 7f05c57fa640 1 -- 172.21.15.31:0/2637886287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+12822 (secure 0 0 0) 0x7f05c00579d0 con 0x7f05c815eb70 2024-01-19T04:57:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:09.887 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:09.191360+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f05b406aed0 msgr2=0x7f05b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f05b406aed0 0x7f05b406d390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f05b8001560 tx=0x7f05b8007990 comp rx=0 tx=0).stop 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 msgr2=0x7f05c80b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80b63b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f05c000ca20 tx=0x7f05c000cee0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 shutdown_connections 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f05b406aed0 0x7f05b406d390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c815eb70 0x7f05c80b63b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f05c815d8a0 0x7f05c80b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 --2- 172.21.15.31:0/2637886287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05c80cca00 0x7f05c81556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 >> 172.21.15.31:0/2637886287 conn(0x7f05c806abe0 msgr2=0x7f05c815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:09.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 shutdown_connections 2024-01-19T04:57:09.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:09.890+0000 7f05ced1f640 1 -- 172.21.15.31:0/2637886287 wait complete. 2024-01-19T04:57:09.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:09.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.005+0000 7fbf51173640 1 Processor -- start 2024-01-19T04:57:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.005+0000 7fbf51173640 1 -- start start 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf51173640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fbf4c0cf570 0x7fbf4c161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf51173640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf4c15d780 0x7fbf4c161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf51173640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c0cf9f0 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf51173640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c0cfcd0 con 0x7fbf4c15d780 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf51173640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c0cffb0 con 0x7fbf4c0cf570 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4b59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf4c15d780 0x7fbf4c161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4b59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf4c15d780 0x7fbf4c161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47602/0 (socket says 172.21.15.31:47602) 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4b59e640 1 -- 172.21.15.31:0/1072676745 learned_addr learned my addr 172.21.15.31:0/1072676745 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4a59c640 1 --2- 172.21.15.31:0/1072676745 >> v2:172.21.15.137:3300/0 conn(0x7fbf4c0cf570 0x7fbf4c161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4b59e640 1 -- 172.21.15.31:0/1072676745 >> v2:172.21.15.137:3300/0 conn(0x7fbf4c0cf570 msgr2=0x7fbf4c161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4b59e640 1 --2- 172.21.15.31:0/1072676745 >> v2:172.21.15.137:3300/0 conn(0x7fbf4c0cf570 0x7fbf4c161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4b59e640 1 -- 172.21.15.31:0/1072676745 >> v1:172.21.15.137:6789/0 conn(0x7fbf4c15d3a0 legacy=0x7fbf4c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4b59e640 1 -- 172.21.15.31:0/1072676745 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0d0290 con 0x7fbf4c15d780 2024-01-19T04:57:10.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.006+0000 7fbf4a59c640 1 --2- 172.21.15.31:0/1072676745 >> v2:172.21.15.137:3300/0 conn(0x7fbf4c0cf570 0x7fbf4c161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.007+0000 7fbf4b59e640 1 --2- 172.21.15.31:0/1072676745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d780 0x7fbf4c161690 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b71b0 tx=0x7fbf3c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7306186b6109a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.007+0000 7fbf49d9b640 1 -- 172.21.15.31:0/1072676745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c02cb20 con 0x7fbf4c15d780 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.007+0000 7fbf49d9b640 1 -- 172.21.15.31:0/1072676745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c02cc80 con 0x7fbf4c15d780 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.007+0000 7fbf49d9b640 1 -- 172.21.15.31:0/1072676745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0343f0 con 0x7fbf4c15d780 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 -- 172.21.15.31:0/1072676745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d780 msgr2=0x7fbf4c161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 --2- 172.21.15.31:0/1072676745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d780 0x7fbf4c161690 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b71b0 tx=0x7fbf3c02bf80 comp rx=0 tx=0).stop 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 -- 172.21.15.31:0/1072676745 shutdown_connections 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 --2- 172.21.15.31:0/1072676745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d780 0x7fbf4c161690 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 --2- 172.21.15.31:0/1072676745 >> v2:172.21.15.137:3300/0 conn(0x7fbf4c0cf570 0x7fbf4c161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 -- 172.21.15.31:0/1072676745 >> 172.21.15.31:0/1072676745 conn(0x7fbf4c0d1100 msgr2=0x7fbf4c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 -- 172.21.15.31:0/1072676745 shutdown_connections 2024-01-19T04:57:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.008+0000 7fbf51173640 1 -- 172.21.15.31:0/1072676745 wait complete. 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.009+0000 7fbf51173640 1 Processor -- start 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.009+0000 7fbf51173640 1 -- start start 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf51173640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf4c0cf570 0x7fbf4c14b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf51173640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d3a0 0x7fbf4c149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf51173640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf4c15d780 0x7fbf4c1473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf51173640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c14b9f0 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf51173640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c145d90 con 0x7fbf4c0cf570 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf51173640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c146050 con 0x7fbf4c15d780 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf4a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d3a0 0x7fbf4c149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf4a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d3a0 0x7fbf4c149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47604/0 (socket says 172.21.15.31:47604) 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf4a59c640 1 -- 172.21.15.31:0/687802599 learned_addr learned my addr 172.21.15.31:0/687802599 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.010+0000 7fbf4ad9d640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf4c0cf570 0x7fbf4c14b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf4b59e640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf4c15d780 0x7fbf4c1473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf4a59c640 1 -- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf4c15d780 msgr2=0x7fbf4c1473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf4a59c640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf4c15d780 0x7fbf4c1473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf4a59c640 1 -- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf4c0cf570 msgr2=0x7fbf4c14b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf4a59c640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf4c0cf570 0x7fbf4c14b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf4a59c640 1 -- 172.21.15.31:0/687802599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c009b30 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf4a59c640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d3a0 0x7fbf4c149540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000e810 tx=0x7fbf4000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf337fe640 1 -- 172.21.15.31:0/687802599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40004030 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf337fe640 1 -- 172.21.15.31:0/687802599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf40004190 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c146330 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf337fe640 1 -- 172.21.15.31:0/687802599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40004520 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.011+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c1468f0 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.012+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c074900 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.012+0000 7fbf337fe640 1 -- 172.21.15.31:0/687802599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fbf4000c6d0 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.013+0000 7fbf337fe640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf3806aed0 0x7fbf3806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.013+0000 7fbf337fe640 1 -- 172.21.15.31:0/687802599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf40088ee0 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.013+0000 7fbf4ad9d640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf3806aed0 0x7fbf3806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.013+0000 7fbf4ad9d640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf3806aed0 0x7fbf3806d390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf440016d0 tx=0x7fbf44009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.017+0000 7fbf337fe640 1 -- 172.21.15.31:0/687802599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbf40054100 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.193+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf4c067030 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.194+0000 7fbf337fe640 1 -- 172.21.15.31:0/687802599 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf40057720 con 0x7fbf4c15d3a0 2024-01-19T04:57:10.193 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf3806aed0 msgr2=0x7fbf3806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf3806aed0 0x7fbf3806d390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf440016d0 tx=0x7fbf44009040 comp rx=0 tx=0).stop 2024-01-19T04:57:10.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d3a0 msgr2=0x7fbf4c149540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d3a0 0x7fbf4c149540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000e810 tx=0x7fbf4000ecd0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 shutdown_connections 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fbf3806aed0 0x7fbf3806d390 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbf4c15d780 0x7fbf4c1473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf4c15d3a0 0x7fbf4c149540 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 --2- 172.21.15.31:0/687802599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf4c0cf570 0x7fbf4c14b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 >> 172.21.15.31:0/687802599 conn(0x7fbf4c0d1100 msgr2=0x7fbf4c06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 shutdown_connections 2024-01-19T04:57:10.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.196+0000 7fbf51173640 1 -- 172.21.15.31:0/687802599 wait complete. 2024-01-19T04:57:10.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-19T04:57:10.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.308+0000 7f3dad0ec640 1 Processor -- start 2024-01-19T04:57:10.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.308+0000 7f3dad0ec640 1 -- start start 2024-01-19T04:57:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.308+0000 7f3dad0ec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3da8160a10 0x7f3da80b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.308+0000 7f3dad0ec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da8160df0 con 0x7f3da80d2e50 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.308+0000 7f3dad0ec640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da80d1db0 con 0x7f3da80b4d00 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.308+0000 7f3dad0ec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da80d2090 con 0x7f3da8160a10 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da6d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3da8160a10 0x7f3da80b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da659c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3da80d2e50 0x7f3da80d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60634/0 (socket says 172.21.15.31:60634) 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da659c640 1 -- 172.21.15.31:0/554500843 learned_addr learned my addr 172.21.15.31:0/554500843 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da5d9b640 1 -- 172.21.15.31:0/554500843 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325264314 0 0) 0x7f3da8160df0 con 0x7f3da80d2e50 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da5d9b640 1 -- 172.21.15.31:0/554500843 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d90003520 con 0x7f3da80d2e50 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da6d9d640 1 -- 172.21.15.31:0/554500843 >> v1:172.21.15.137:6789/0 conn(0x7f3da80b4d00 legacy=0x7f3da80b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da6d9d640 1 -- 172.21.15.31:0/554500843 >> v1:172.21.15.31:6789/0 conn(0x7f3da80d2e50 legacy=0x7f3da80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.309+0000 7f3da6d9d640 1 -- 172.21.15.31:0/554500843 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80d2370 con 0x7f3da8160a10 2024-01-19T04:57:10.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3da6d9d640 1 --2- 172.21.15.31:0/554500843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da8160a10 0x7f3da80b8ea0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3d980097a0 tx=0x7f3d9802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a500063e35363e5c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3da5d9b640 1 -- 172.21.15.31:0/554500843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98030610 con 0x7f3da8160a10 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3da5d9b640 1 -- 172.21.15.31:0/554500843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d98030770 con 0x7f3da8160a10 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3da5d9b640 1 -- 172.21.15.31:0/554500843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98030b00 con 0x7f3da8160a10 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3dad0ec640 1 -- 172.21.15.31:0/554500843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da8160a10 msgr2=0x7f3da80b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/554500843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da8160a10 0x7f3da80b8ea0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3d980097a0 tx=0x7f3d9802b960 comp rx=0 tx=0).stop 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3dad0ec640 1 -- 172.21.15.31:0/554500843 shutdown_connections 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/554500843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da8160a10 0x7f3da80b8ea0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3dad0ec640 1 -- 172.21.15.31:0/554500843 >> 172.21.15.31:0/554500843 conn(0x7f3da8072310 msgr2=0x7f3da80cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3dad0ec640 1 -- 172.21.15.31:0/554500843 shutdown_connections 2024-01-19T04:57:10.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.310+0000 7f3dad0ec640 1 -- 172.21.15.31:0/554500843 wait complete. 2024-01-19T04:57:10.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.312+0000 7f3dad0ec640 1 Processor -- start 2024-01-19T04:57:10.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.312+0000 7f3dad0ec640 1 -- start start 2024-01-19T04:57:10.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.312+0000 7f3dad0ec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3da80b4d00 0x7f3da8154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.312+0000 7f3dad0ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da80d2e50 0x7f3da815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3dad0ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8160a10 0x7f3da815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3dad0ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da815a590 con 0x7f3da80d2e50 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3dad0ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80cd200 con 0x7f3da8160a10 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3dad0ec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8159910 con 0x7f3da80b4d00 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da80d2e50 0x7f3da815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da80d2e50 0x7f3da815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47616/0 (socket says 172.21.15.31:47616) 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 -- 172.21.15.31:0/1189615811 learned_addr learned my addr 172.21.15.31:0/1189615811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da759e640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8160a10 0x7f3da815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da6d9d640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3da80b4d00 0x7f3da8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 -- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3da80b4d00 msgr2=0x7f3da8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3da80b4d00 0x7f3da8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 -- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8160a10 msgr2=0x7f3da815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8160a10 0x7f3da815b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 -- 172.21.15.31:0/1189615811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d98009400 con 0x7f3da80d2e50 2024-01-19T04:57:10.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da759e640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8160a10 0x7f3da815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:10.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3da659c640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da80d2e50 0x7f3da815b140 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c4d0 tx=0x7f3d9c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3d8f7fe640 1 -- 172.21.15.31:0/1189615811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c005ce0 con 0x7f3da80d2e50 2024-01-19T04:57:10.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3d8f7fe640 1 -- 172.21.15.31:0/1189615811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c005e40 con 0x7f3da80d2e50 2024-01-19T04:57:10.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3d8f7fe640 1 -- 172.21.15.31:0/1189615811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c007270 con 0x7f3da80d2e50 2024-01-19T04:57:10.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da8159bf0 con 0x7f3da80d2e50 2024-01-19T04:57:10.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.313+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da815a1b0 con 0x7f3da80d2e50 2024-01-19T04:57:10.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.315+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da815a6d0 con 0x7f3da80d2e50 2024-01-19T04:57:10.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.315+0000 7f3d8f7fe640 1 -- 172.21.15.31:0/1189615811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3d9c007490 con 0x7f3da80d2e50 2024-01-19T04:57:10.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.315+0000 7f3d8f7fe640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3d9406aed0 0x7f3d9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.315+0000 7f3d8f7fe640 1 -- 172.21.15.31:0/1189615811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3d9c088730 con 0x7f3da80d2e50 2024-01-19T04:57:10.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.316+0000 7f3da6d9d640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3d9406aed0 0x7f3d9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.316+0000 7f3da6d9d640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3d9406aed0 0x7f3d9406d390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d98036000 tx=0x7f3d9803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.319+0000 7f3d8f7fe640 1 -- 172.21.15.31:0/1189615811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3da815a6d0 con 0x7f3da80d2e50 2024-01-19T04:57:10.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.502+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3da815a6d0 con 0x7f3da80d2e50 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.502+0000 7f3d8f7fe640 1 -- 172.21.15.31:0/1189615811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3da815a6d0 con 0x7f3da80d2e50 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:10.502 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3d9406aed0 msgr2=0x7f3d9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3d9406aed0 0x7f3d9406d390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d98036000 tx=0x7f3d9803b040 comp rx=0 tx=0).stop 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da80d2e50 msgr2=0x7f3da815b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da80d2e50 0x7f3da815b140 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c4d0 tx=0x7f3d9c00c990 comp rx=0 tx=0).stop 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 shutdown_connections 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3d9406aed0 0x7f3d9406d390 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8160a10 0x7f3da815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3da80d2e50 0x7f3da815b140 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 --2- 172.21.15.31:0/1189615811 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3da80b4d00 0x7f3da8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 >> 172.21.15.31:0/1189615811 conn(0x7f3da8072310 msgr2=0x7f3da80d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:10.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 shutdown_connections 2024-01-19T04:57:10.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.504+0000 7f3dad0ec640 1 -- 172.21.15.31:0/1189615811 wait complete. 2024-01-19T04:57:10.515 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-19T04:57:10.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-19T04:57:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.620+0000 7f3e688ee640 1 Processor -- start 2024-01-19T04:57:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.620+0000 7f3e688ee640 1 -- start start 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e688ee640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e6415d880 0x7f3e640d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e688ee640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e640d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e688ee640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e640d2050 con 0x7f3e640d0890 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e688ee640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e640d0d20 con 0x7f3e6415d880 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e688ee640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640d2330 con 0x7f3e640b8d40 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e6259c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e6415d880 0x7f3e640d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e62d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e640d0890 0x7f3e640b4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60646/0 (socket says 172.21.15.31:60646) 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e62d9d640 1 -- 172.21.15.31:0/1606268395 learned_addr learned my addr 172.21.15.31:0/1606268395 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.621+0000 7f3e61d9b640 1 --2- 172.21.15.31:0/1606268395 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e640d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.622+0000 7f3e6259c640 1 -- 172.21.15.31:0/1606268395 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 msgr2=0x7f3e640d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:10.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.622+0000 7f3e6259c640 1 --2- 172.21.15.31:0/1606268395 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e640d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.622+0000 7f3e6259c640 1 -- 172.21.15.31:0/1606268395 >> v1:172.21.15.31:6789/0 conn(0x7f3e640d0890 legacy=0x7f3e640b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.622+0000 7f3e6159a640 1 -- 172.21.15.31:0/1606268395 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909402015 0 0) 0x7f3e640d2050 con 0x7f3e640d0890 2024-01-19T04:57:10.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.622+0000 7f3e6259c640 1 -- 172.21.15.31:0/1606268395 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640d2610 con 0x7f3e6415d880 2024-01-19T04:57:10.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.622+0000 7f3e6259c640 1 --2- 172.21.15.31:0/1606268395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6415d880 0x7f3e640d1800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0097a0 tx=0x7f3e5c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5211f7cf6b022b43 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.622+0000 7f3e6159a640 1 -- 172.21.15.31:0/1606268395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c030610 con 0x7f3e6415d880 2024-01-19T04:57:10.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e6159a640 1 -- 172.21.15.31:0/1606268395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c030770 con 0x7f3e6415d880 2024-01-19T04:57:10.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e6159a640 1 -- 172.21.15.31:0/1606268395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c030b00 con 0x7f3e6415d880 2024-01-19T04:57:10.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 -- 172.21.15.31:0/1606268395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6415d880 msgr2=0x7f3e640d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 --2- 172.21.15.31:0/1606268395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6415d880 0x7f3e640d1800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0097a0 tx=0x7f3e5c02b960 comp rx=0 tx=0).stop 2024-01-19T04:57:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 -- 172.21.15.31:0/1606268395 shutdown_connections 2024-01-19T04:57:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 --2- 172.21.15.31:0/1606268395 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e640d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 --2- 172.21.15.31:0/1606268395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6415d880 0x7f3e640d1800 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 -- 172.21.15.31:0/1606268395 >> 172.21.15.31:0/1606268395 conn(0x7f3e640720d0 msgr2=0x7f3e640724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 -- 172.21.15.31:0/1606268395 shutdown_connections 2024-01-19T04:57:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.623+0000 7f3e688ee640 1 -- 172.21.15.31:0/1606268395 wait complete. 2024-01-19T04:57:10.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.624+0000 7f3e688ee640 1 Processor -- start 2024-01-19T04:57:10.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e688ee640 1 -- start start 2024-01-19T04:57:10.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e688ee640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e6415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e688ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e640d0890 0x7f3e6415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e688ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e6415d880 0x7f3e64159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e688ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415f8b0 con 0x7f3e640d0890 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e688ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415fb90 con 0x7f3e6415d880 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e688ee640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415fe70 con 0x7f3e640b8d40 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e61d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e640d0890 0x7f3e6415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e61d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e640d0890 0x7f3e6415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47642/0 (socket says 172.21.15.31:47642) 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.625+0000 7f3e61d9b640 1 -- 172.21.15.31:0/3917043150 learned_addr learned my addr 172.21.15.31:0/3917043150 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e6259c640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e6415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e62d9d640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e6415d880 0x7f3e64159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e61d9b640 1 -- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 msgr2=0x7f3e6415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e61d9b640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e6415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e61d9b640 1 -- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e6415d880 msgr2=0x7f3e64159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e61d9b640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e6415d880 0x7f3e64159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e61d9b640 1 -- 172.21.15.31:0/3917043150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c009400 con 0x7f3e640d0890 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e6259c640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e6415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e61d9b640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e640d0890 0x7f3e6415f080 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00d800 tx=0x7f3e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e5affd640 1 -- 172.21.15.31:0/3917043150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00b6c0 con 0x7f3e640d0890 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e5affd640 1 -- 172.21.15.31:0/3917043150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c00b820 con 0x7f3e640d0890 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e64160150 con 0x7f3e640d0890 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e5affd640 1 -- 172.21.15.31:0/3917043150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00bbb0 con 0x7f3e640d0890 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.626+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6415f5c0 con 0x7f3e640d0890 2024-01-19T04:57:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.627+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e640696e0 con 0x7f3e640d0890 2024-01-19T04:57:10.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.627+0000 7f3e5affd640 1 -- 172.21.15.31:0/3917043150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3e4c00bd10 con 0x7f3e640d0890 2024-01-19T04:57:10.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.628+0000 7f3e5affd640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e4406aed0 0x7f3e4406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:10.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.628+0000 7f3e5affd640 1 -- 172.21.15.31:0/3917043150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e4c089780 con 0x7f3e640d0890 2024-01-19T04:57:10.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.628+0000 7f3e6259c640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e4406aed0 0x7f3e4406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:10.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.629+0000 7f3e6259c640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e4406aed0 0x7f3e4406d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c036000 tx=0x7f3e5c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:10.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.632+0000 7f3e5affd640 1 -- 172.21.15.31:0/3917043150 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3e4c054a30 con 0x7f3e640d0890 2024-01-19T04:57:10.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:10.811+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3e640b8840 con 0x7f3e640d0890 2024-01-19T04:57:11.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.508+0000 7f3e5affd640 1 -- 172.21.15.31:0/3917043150 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3e4c058050 con 0x7f3e640d0890 2024-01-19T04:57:11.508 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-01-19T04:57:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e4406aed0 msgr2=0x7f3e4406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e4406aed0 0x7f3e4406d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c036000 tx=0x7f3e5c03b040 comp rx=0 tx=0).stop 2024-01-19T04:57:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e640d0890 msgr2=0x7f3e6415f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e640d0890 0x7f3e6415f080 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00d800 tx=0x7f3e4c00dcc0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 shutdown_connections 2024-01-19T04:57:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e4406aed0 0x7f3e4406d390 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e6415d880 0x7f3e64159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e640d0890 0x7f3e6415f080 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 --2- 172.21.15.31:0/3917043150 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e640b8d40 0x7f3e6415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.511+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 >> 172.21.15.31:0/3917043150 conn(0x7f3e640720d0 msgr2=0x7f3e640d33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.512+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 shutdown_connections 2024-01-19T04:57:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.513+0000 7f3e688ee640 1 -- 172.21.15.31:0/3917043150 wait complete. 2024-01-19T04:57:11.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-19T04:57:11.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.640+0000 7f01c3089640 1 Processor -- start 2024-01-19T04:57:11.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.640+0000 7f01c3089640 1 -- start start 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.640+0000 7f01c3089640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01bc15d720 0x7f01bc0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.640+0000 7f01c3089640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc0cf160 con 0x7f01bc0d3ea0 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.640+0000 7f01c3089640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc0ce8c0 con 0x7f01bc0d1c20 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.640+0000 7f01c3089640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc0ceba0 con 0x7f01bc15d720 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f01bc0d3ea0 0x7f01bc0d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60648/0 (socket says 172.21.15.31:60648) 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bbfff640 1 -- 172.21.15.31:0/3169951757 learned_addr learned my addr 172.21.15.31:0/3169951757 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01c0e25640 1 --2- 172.21.15.31:0/3169951757 >> v2:172.21.15.137:3300/0 conn(0x7f01bc15d720 0x7f01bc0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559059127 0 0) 0x7f01bc0cf160 con 0x7f01bc0d3ea0 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ac003520 con 0x7f01bc0d3ea0 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1248074422 0 0) 0x7f01ac003520 con 0x7f01bc0d3ea0 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 >> v2:172.21.15.137:3300/0 conn(0x7f01bc15d720 msgr2=0x7f01bc0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 --2- 172.21.15.31:0/3169951757 >> v2:172.21.15.137:3300/0 conn(0x7f01bc15d720 0x7f01bc0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 >> v1:172.21.15.137:6789/0 conn(0x7f01bc0d1c20 legacy=0x7f01bc0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:11.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc0cee80 con 0x7f01bc0d3ea0 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f01a8001670 con 0x7f01bc0d3ea0 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 >> v1:172.21.15.31:6789/0 conn(0x7f01bc0d3ea0 legacy=0x7f01bc0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac003970 0x7f01ac003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01ac0047c0 0x7f01ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac0052a0 0x7f01ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0ceba0 con 0x7f01ac003970 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac00c170 con 0x7f01ac0052a0 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac00c330 con 0x7f01ac0047c0 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01c0e25640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac003970 0x7f01ac003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01bbfff640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01ac0047c0 0x7f01ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.641+0000 7f01c1626640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac0052a0 0x7f01ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c0e25640 1 -- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01ac0047c0 msgr2=0x7f01ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c0e25640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01ac0047c0 0x7f01ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c0e25640 1 -- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac0052a0 msgr2=0x7f01ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:11.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c0e25640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac0052a0 0x7f01ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c0e25640 1 -- 172.21.15.31:0/3169951757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac003520 con 0x7f01ac003970 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01bbfff640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01ac0047c0 0x7f01ac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c1626640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac0052a0 0x7f01ac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c1626640 1 -- 172.21.15.31:0/3169951757 reap_dead start 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01c0e25640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac003970 0x7f01ac003db0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009ac0 tx=0x7f01ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a78aba9e6c3355 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b002c6b0 con 0x7f01ac003970 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b002c810 con 0x7f01ac003970 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.642+0000 7f01bb7fe640 1 -- 172.21.15.31:0/3169951757 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b002cba0 con 0x7f01ac003970 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.643+0000 7f01c3089640 1 -- 172.21.15.31:0/3169951757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac003970 msgr2=0x7f01ac003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.643+0000 7f01c3089640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac003970 0x7f01ac003db0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009ac0 tx=0x7f01ac0057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.643+0000 7f01c3089640 1 -- 172.21.15.31:0/3169951757 shutdown_connections 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.643+0000 7f01c3089640 1 --2- 172.21.15.31:0/3169951757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac003970 0x7f01ac003db0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.643+0000 7f01c3089640 1 -- 172.21.15.31:0/3169951757 >> 172.21.15.31:0/3169951757 conn(0x7f01bc0d3310 msgr2=0x7f01bc0d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.643+0000 7f01c3089640 1 -- 172.21.15.31:0/3169951757 shutdown_connections 2024-01-19T04:57:11.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.643+0000 7f01c3089640 1 -- 172.21.15.31:0/3169951757 wait complete. 2024-01-19T04:57:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.644+0000 7f01c3089640 1 Processor -- start 2024-01-19T04:57:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.644+0000 7f01c3089640 1 -- start start 2024-01-19T04:57:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c3089640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac003970 0x7f01bc0b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c3089640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01bc15ae70 0x7f01bc1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c3089640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01bc1f44e0 0x7f01bc0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c3089640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1f3390 con 0x7f01bc1f44e0 2024-01-19T04:57:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c3089640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1f6e30 con 0x7f01ac003970 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c3089640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1f7110 con 0x7f01bc15ae70 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01bc1f44e0 0x7f01bc0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01bc1f44e0 0x7f01bc0c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47662/0 (socket says 172.21.15.31:47662) 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 -- 172.21.15.31:0/562566833 learned_addr learned my addr 172.21.15.31:0/562566833 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01bbfff640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01bc15ae70 0x7f01bc1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c0e25640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac003970 0x7f01bc0b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 -- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01bc15ae70 msgr2=0x7f01bc1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01bc15ae70 0x7f01bc1f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 -- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac003970 msgr2=0x7f01bc0b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac003970 0x7f01bc0b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.645+0000 7f01c1626640 1 -- 172.21.15.31:0/562566833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc1f73f0 con 0x7f01bc1f44e0 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01bbfff640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01bc15ae70 0x7f01bc1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01c0e25640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac003970 0x7f01bc0b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01c1626640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01bc1f44e0 0x7f01bc0c9010 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f01bc1611c0 tx=0x7f01b400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01b97fa640 1 -- 172.21.15.31:0/562566833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b400eed0 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01b97fa640 1 -- 172.21.15.31:0/562566833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b4003970 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc1f2ff0 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01b97fa640 1 -- 172.21.15.31:0/562566833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4003d20 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.646+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc15b2f0 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.647+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc15db00 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.647+0000 7f01b97fa640 1 -- 172.21.15.31:0/562566833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f01b40050d0 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.648+0000 7f01b97fa640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01a006aed0 0x7f01a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.648+0000 7f01b97fa640 1 -- 172.21.15.31:0/562566833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f01b4089290 con 0x7f01bc1f44e0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.648+0000 7f01c0e25640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01a006aed0 0x7f01a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.649+0000 7f01c0e25640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01a006aed0 0x7f01a006d390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f01b0036000 tx=0x7f01b0030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.652+0000 7f01b97fa640 1 -- 172.21.15.31:0/562566833 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f01b40542e0 con 0x7f01bc1f44e0 2024-01-19T04:57:11.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:11.831+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f01bc1f31e0 con 0x7f01bc1f44e0 2024-01-19T04:57:12.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.513+0000 7f01b97fa640 1 -- 172.21.15.31:0/562566833 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) v1 ==== 170+0+0 (secure 0 0 0) 0x7f01b4057900 con 0x7f01bc1f44e0 2024-01-19T04:57:12.513 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01a006aed0 msgr2=0x7f01a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01a006aed0 0x7f01a006d390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f01b0036000 tx=0x7f01b0030000 comp rx=0 tx=0).stop 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01bc1f44e0 msgr2=0x7f01bc0c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01bc1f44e0 0x7f01bc0c9010 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f01bc1611c0 tx=0x7f01b400eb90 comp rx=0 tx=0).stop 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 shutdown_connections 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f01a006aed0 0x7f01a006d390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01bc1f44e0 0x7f01bc0c9010 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01bc15ae70 0x7f01bc1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 --2- 172.21.15.31:0/562566833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ac003970 0x7f01bc0b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 >> 172.21.15.31:0/562566833 conn(0x7f01bc0d3310 msgr2=0x7f01bc0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 shutdown_connections 2024-01-19T04:57:12.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.516+0000 7f01c3089640 1 -- 172.21.15.31:0/562566833 wait complete. 2024-01-19T04:57:12.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-19T04:57:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.652+0000 7f4ef0690640 1 Processor -- start 2024-01-19T04:57:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.652+0000 7f4ef0690640 1 -- start start 2024-01-19T04:57:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.653+0000 7f4ef0690640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 0x7f4ee806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.653+0000 7f4ef0690640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4ee806aaf0 0x7f4ee8071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.653+0000 7f4ef0690640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee80c6130 con 0x7f4ee815d340 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.653+0000 7f4ef0690640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee806af80 con 0x7f4ee806aaf0 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.653+0000 7f4ef0690640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80ce630 con 0x7f4ee80cc660 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.653+0000 7f4eee42c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4ee815d340 0x7f4ee80cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60662/0 (socket says 172.21.15.31:60662) 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.653+0000 7f4eee42c640 1 -- 172.21.15.31:0/3261031658 learned_addr learned my addr 172.21.15.31:0/3261031658 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800232724 0 0) 0x7f4ee80c6130 con 0x7f4ee815d340 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eedc2b640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 0x7f4ee806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed4003520 con 0x7f4ee815d340 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eeec2d640 1 --2- 172.21.15.31:0/3261031658 >> v2:172.21.15.137:3300/0 conn(0x7f4ee806aaf0 0x7f4ee8071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3311649132 0 0) 0x7f4ed4003520 con 0x7f4ee815d340 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 msgr2=0x7f4ee806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 0x7f4ee806a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 >> v2:172.21.15.137:3300/0 conn(0x7f4ee806aaf0 msgr2=0x7f4ee8071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 --2- 172.21.15.31:0/3261031658 >> v2:172.21.15.137:3300/0 conn(0x7f4ee806aaf0 0x7f4ee8071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80c5520 con 0x7f4ee815d340 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f4ee4001670 con 0x7f4ee815d340 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 >> v1:172.21.15.31:6789/0 conn(0x7f4ee815d340 legacy=0x7f4ee80cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed4003970 0x7f4ed4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed40047c0 0x7f4ed4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 0x7f4ed4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee806af80 con 0x7f4ed4003970 2024-01-19T04:57:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80ce630 con 0x7f4ed40047c0 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed400bb90 con 0x7f4ee80cc660 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eee42c640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed4003970 0x7f4ed4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eedc2b640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed40047c0 0x7f4ed4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eeec2d640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 0x7f4ed4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eee42c640 1 -- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 msgr2=0x7f4ed4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eee42c640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80cc660 0x7f4ed4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eee42c640 1 -- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed40047c0 msgr2=0x7f4ed4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eee42c640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed40047c0 0x7f4ed4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eee42c640 1 -- 172.21.15.31:0/3261031658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4003520 con 0x7f4ed4003970 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.654+0000 7f4eedc2b640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed40047c0 0x7f4ed4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.655+0000 7f4eee42c640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed4003970 0x7f4ed4003d90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c290 tx=0x7f4ed40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7877e91e3249a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.655+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4005ad0 con 0x7f4ed4003970 2024-01-19T04:57:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.655+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4005c30 con 0x7f4ed4003970 2024-01-19T04:57:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.655+0000 7f4eed42a640 1 -- 172.21.15.31:0/3261031658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee402c8d0 con 0x7f4ed4003970 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.656+0000 7f4ef0690640 1 -- 172.21.15.31:0/3261031658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed4003970 msgr2=0x7f4ed4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.656+0000 7f4ef0690640 1 --2- 172.21.15.31:0/3261031658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed4003970 0x7f4ed4003d90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c290 tx=0x7f4ed40057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.656+0000 7f4eeec2d640 1 -- 172.21.15.31:0/3261031658 reap_dead start 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.656+0000 7f4ef0690640 1 -- 172.21.15.31:0/3261031658 shutdown_connections 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.656+0000 7f4ef0690640 1 -- 172.21.15.31:0/3261031658 >> 172.21.15.31:0/3261031658 conn(0x7f4ee80b1c50 msgr2=0x7f4ee80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.656+0000 7f4ef0690640 1 -- 172.21.15.31:0/3261031658 shutdown_connections 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.656+0000 7f4ef0690640 1 -- 172.21.15.31:0/3261031658 wait complete. 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.657+0000 7f4ef0690640 1 Processor -- start 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.657+0000 7f4ef0690640 1 -- start start 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4ef0690640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4003970 0x7f4ee815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4ef0690640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee81f2de0 0x7f4ee81f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4ef0690640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee81f37a0 0x7f4ee81f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4ef0690640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b5290 con 0x7f4ee81f37a0 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4ef0690640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b4610 con 0x7f4ed4003970 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4ef0690640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b48f0 con 0x7f4ee81f2de0 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee81f37a0 0x7f4ee81f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee81f37a0 0x7f4ee81f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47686/0 (socket says 172.21.15.31:47686) 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 -- 172.21.15.31:0/1307542922 learned_addr learned my addr 172.21.15.31:0/1307542922 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eedc2b640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee81f2de0 0x7f4ee81f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eee42c640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4003970 0x7f4ee815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 -- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee81f2de0 msgr2=0x7f4ee81f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee81f2de0 0x7f4ee81f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 -- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4003970 msgr2=0x7f4ee815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4003970 0x7f4ee815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 -- 172.21.15.31:0/1307542922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80b4bd0 con 0x7f4ee81f37a0 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4eeec2d640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee81f37a0 0x7f4ee81f4280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4ee806ffc0 tx=0x7f4ee0007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.658+0000 7f4edeffd640 1 -- 172.21.15.31:0/1307542922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0003ea0 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.659+0000 7f4edeffd640 1 -- 172.21.15.31:0/1307542922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee000f050 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.659+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee80b4eb0 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.659+0000 7f4edeffd640 1 -- 172.21.15.31:0/1307542922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0014650 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.659+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee81f3c00 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.659+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee81f3fa0 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.660+0000 7f4edeffd640 1 -- 172.21.15.31:0/1307542922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4ee00147b0 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.660+0000 7f4edeffd640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4ecc06aed0 0x7f4ecc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.660+0000 7f4edeffd640 1 -- 172.21.15.31:0/1307542922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4ee00897f0 con 0x7f4ee81f37a0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.661+0000 7f4eee42c640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4ecc06aed0 0x7f4ecc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.661+0000 7f4eee42c640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4ecc06aed0 0x7f4ecc06d390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4034000 tx=0x7f4ee4007000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.665+0000 7f4edeffd640 1 -- 172.21.15.31:0/1307542922 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4ee81f3fa0 con 0x7f4ee81f37a0 2024-01-19T04:57:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:12.878+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4ee81f3fa0 con 0x7f4ee81f37a0 2024-01-19T04:57:13.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.515+0000 7f4edeffd640 1 -- 172.21.15.31:0/1307542922 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4ee81f3fa0 con 0x7f4ee81f37a0 2024-01-19T04:57:13.515 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 7 and data pool 8 2024-01-19T04:57:13.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.519+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4ecc06aed0 msgr2=0x7f4ecc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.519+0000 7f4ef0690640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4ecc06aed0 0x7f4ecc06d390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4034000 tx=0x7f4ee4007000 comp rx=0 tx=0).stop 2024-01-19T04:57:13.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee81f37a0 msgr2=0x7f4ee81f4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee81f37a0 0x7f4ee81f4280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4ee806ffc0 tx=0x7f4ee0007700 comp rx=0 tx=0).stop 2024-01-19T04:57:13.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 shutdown_connections 2024-01-19T04:57:13.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4ecc06aed0 0x7f4ecc06d390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee81f37a0 0x7f4ee81f4280 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee81f2de0 0x7f4ee81f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 --2- 172.21.15.31:0/1307542922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4003970 0x7f4ee815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.520+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 >> 172.21.15.31:0/1307542922 conn(0x7f4ee80b1c50 msgr2=0x7f4ee80c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:13.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.522+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 shutdown_connections 2024-01-19T04:57:13.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.523+0000 7f4ef0690640 1 -- 172.21.15.31:0/1307542922 wait complete. 2024-01-19T04:57:13.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.662+0000 7f62fbfea640 1 Processor -- start 2024-01-19T04:57:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.662+0000 7f62fbfea640 1 -- start start 2024-01-19T04:57:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fbfea640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f40d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fbfea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62f40d0ef0 0x7f62f40d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fbfea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40d06b0 con 0x7f62f4160bb0 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fbfea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40d18c0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fbfea640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40d1b90 con 0x7f62f40cfd10 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fa587640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f62f40d0ef0 0x7f62f40d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62f9d86640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f62f4160bb0 0x7f62f40b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60664/0 (socket says 172.21.15.31:60664) 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62f9d86640 1 -- 172.21.15.31:0/2195111465 learned_addr learned my addr 172.21.15.31:0/2195111465 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fa587640 1 --2- 172.21.15.31:0/2195111465 >> v2:172.21.15.31:3300/0 conn(0x7f62f40d0ef0 0x7f62f40d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47694/0 (socket says 172.21.15.31:47694) 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62f9585640 1 --2- 172.21.15.31:0/2195111465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f40d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fa587640 1 -- 172.21.15.31:0/2195111465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 msgr2=0x7f62f40d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fa587640 1 --2- 172.21.15.31:0/2195111465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f40d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fa587640 1 -- 172.21.15.31:0/2195111465 >> v1:172.21.15.31:6789/0 conn(0x7f62f4160bb0 legacy=0x7f62f40b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62fa587640 1 -- 172.21.15.31:0/2195111465 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40d1e70 con 0x7f62f40d0ef0 2024-01-19T04:57:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.663+0000 7f62f9585640 1 --2- 172.21.15.31:0/2195111465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.664+0000 7f62fa587640 1 --2- 172.21.15.31:0/2195111465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f40d1380 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009d90 tx=0x7f62ec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a6914c06823cb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.664+0000 7f62f8d84640 1 -- 172.21.15.31:0/2195111465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec02cc50 con 0x7f62f40d0ef0 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.664+0000 7f62f8d84640 1 -- 172.21.15.31:0/2195111465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec02cdb0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.664+0000 7f62f8d84640 1 -- 172.21.15.31:0/2195111465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec034660 con 0x7f62f40d0ef0 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.664+0000 7f62fbfea640 1 -- 172.21.15.31:0/2195111465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 msgr2=0x7f62f40d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.664+0000 7f62fbfea640 1 --2- 172.21.15.31:0/2195111465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f40d1380 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009d90 tx=0x7f62ec02bea0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.665+0000 7f62fbfea640 1 -- 172.21.15.31:0/2195111465 shutdown_connections 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.665+0000 7f62fbfea640 1 --2- 172.21.15.31:0/2195111465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f40d1380 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.665+0000 7f62fbfea640 1 --2- 172.21.15.31:0/2195111465 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.665+0000 7f62fbfea640 1 -- 172.21.15.31:0/2195111465 >> 172.21.15.31:0/2195111465 conn(0x7f62f409c600 msgr2=0x7f62f40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.665+0000 7f62fbfea640 1 -- 172.21.15.31:0/2195111465 shutdown_connections 2024-01-19T04:57:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.665+0000 7f62fbfea640 1 -- 172.21.15.31:0/2195111465 wait complete. 2024-01-19T04:57:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.666+0000 7f62fbfea640 1 Processor -- start 2024-01-19T04:57:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.666+0000 7f62fbfea640 1 -- start start 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.666+0000 7f62fbfea640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f4159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.666+0000 7f62fbfea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.666+0000 7f62fbfea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62f4160bb0 0x7f62f415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62fbfea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40b5a20 con 0x7f62f40d0ef0 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62fbfea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40b4e10 con 0x7f62f4160bb0 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62fbfea640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40b50f0 con 0x7f62f40cfd10 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47696/0 (socket says 172.21.15.31:47696) 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 -- 172.21.15.31:0/3742881722 learned_addr learned my addr 172.21.15.31:0/3742881722 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9d86640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f4159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62fa587640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62f4160bb0 0x7f62f415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 -- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 msgr2=0x7f62f4159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f4159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 -- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62f4160bb0 msgr2=0x7f62f415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62f4160bb0 0x7f62f415e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 -- 172.21.15.31:0/3742881722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec009a70 con 0x7f62f40d0ef0 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9d86640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62f9585640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f415e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c7f0 tx=0x7f62e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62ea7fc640 1 -- 172.21.15.31:0/3742881722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4007a30 con 0x7f62f40d0ef0 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62ea7fc640 1 -- 172.21.15.31:0/3742881722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e4007b90 con 0x7f62f40d0ef0 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f40b53d0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62ea7fc640 1 -- 172.21.15.31:0/3742881722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400ce40 con 0x7f62f40d0ef0 2024-01-19T04:57:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.667+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f415a320 con 0x7f62f40d0ef0 2024-01-19T04:57:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.668+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f40696e0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.669+0000 7f62ea7fc640 1 -- 172.21.15.31:0/3742881722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f62e40113f0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.669+0000 7f62ea7fc640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f62d806aed0 0x7f62d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.669+0000 7f62ea7fc640 1 -- 172.21.15.31:0/3742881722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62e40897b0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.669+0000 7f62f9d86640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f62d806aed0 0x7f62d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:13.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.670+0000 7f62f9d86640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f62d806aed0 0x7f62d806d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000f60 tx=0x7f62f0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.674+0000 7f62ea7fc640 1 -- 172.21.15.31:0/3742881722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f62e4054680 con 0x7f62f40d0ef0 2024-01-19T04:57:13.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:57:13.792 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:57:13.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.852+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62f408d3b0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.854+0000 7f62ea7fc640 1 -- 172.21.15.31:0/3742881722 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f62e4057ca0 con 0x7f62f40d0ef0 2024-01-19T04:57:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:13.853 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":49,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:12.879709+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:13.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f62d806aed0 msgr2=0x7f62d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f62d806aed0 0x7f62d806d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000f60 tx=0x7f62f0009040 comp rx=0 tx=0).stop 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 msgr2=0x7f62f415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f415e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c7f0 tx=0x7f62e400cbf0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 shutdown_connections 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f62d806aed0 0x7f62d806d390 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62f4160bb0 0x7f62f415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f40d0ef0 0x7f62f415e2f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 --2- 172.21.15.31:0/3742881722 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f62f40cfd10 0x7f62f4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 >> 172.21.15.31:0/3742881722 conn(0x7f62f409c600 msgr2=0x7f62f415cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 shutdown_connections 2024-01-19T04:57:13.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:13.856+0000 7f62fbfea640 1 -- 172.21.15.31:0/3742881722 wait complete. 2024-01-19T04:57:13.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:14.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.008+0000 7f75ff29b640 1 Processor -- start 2024-01-19T04:57:14.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.008+0000 7f75ff29b640 1 -- start start 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75ff29b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f815d8a0 0x7f75f806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75ff29b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75f815eb70 0x7f75f80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75ff29b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f75f80cca00 0x7f75f815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75ff29b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f806bf30 con 0x7f75f815eb70 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75ff29b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f80cb9e0 con 0x7f75f80cca00 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75ff29b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80cbcc0 con 0x7f75f815d8a0 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75f815eb70 0x7f75f80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75f815eb70 0x7f75f80cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47704/0 (socket says 172.21.15.31:47704) 2024-01-19T04:57:14.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 -- 172.21.15.31:0/3823930577 learned_addr learned my addr 172.21.15.31:0/3823930577 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fd037640 1 --2- 172.21.15.31:0/3823930577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f815d8a0 0x7f75f806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fd838640 1 --2- 172.21.15.31:0/3823930577 >> v2:172.21.15.137:3300/0 conn(0x7f75f80cca00 0x7f75f815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 -- 172.21.15.31:0/3823930577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f815d8a0 msgr2=0x7f75f806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 --2- 172.21.15.31:0/3823930577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f815d8a0 0x7f75f806b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 -- 172.21.15.31:0/3823930577 >> v2:172.21.15.137:3300/0 conn(0x7f75f80cca00 msgr2=0x7f75f815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 --2- 172.21.15.31:0/3823930577 >> v2:172.21.15.137:3300/0 conn(0x7f75f80cca00 0x7f75f815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.009+0000 7f75fc836640 1 -- 172.21.15.31:0/3823930577 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80cbfa0 con 0x7f75f815eb70 2024-01-19T04:57:14.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.010+0000 7f75fc836640 1 --2- 172.21.15.31:0/3823930577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f80cc4c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f75f80d2220 tx=0x7f75ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79653297e051000 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:14.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.010+0000 7f75f7fff640 1 -- 172.21.15.31:0/3823930577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0317e0 con 0x7f75f815eb70 2024-01-19T04:57:14.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.010+0000 7f75f7fff640 1 -- 172.21.15.31:0/3823930577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec031940 con 0x7f75f815eb70 2024-01-19T04:57:14.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.010+0000 7f75f7fff640 1 -- 172.21.15.31:0/3823930577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec031cd0 con 0x7f75f815eb70 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 -- 172.21.15.31:0/3823930577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 msgr2=0x7f75f80cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3823930577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f80cc4c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f75f80d2220 tx=0x7f75ec02b880 comp rx=0 tx=0).stop 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 -- 172.21.15.31:0/3823930577 shutdown_connections 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3823930577 >> v2:172.21.15.137:3300/0 conn(0x7f75f80cca00 0x7f75f815f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3823930577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f80cc4c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3823930577 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f815d8a0 0x7f75f806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 -- 172.21.15.31:0/3823930577 >> 172.21.15.31:0/3823930577 conn(0x7f75f806abe0 msgr2=0x7f75f8066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 -- 172.21.15.31:0/3823930577 shutdown_connections 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.011+0000 7f75ff29b640 1 -- 172.21.15.31:0/3823930577 wait complete. 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.012+0000 7f75ff29b640 1 Processor -- start 2024-01-19T04:57:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.012+0000 7f75ff29b640 1 -- start start 2024-01-19T04:57:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75ff29b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f80cca00 0x7f75f80b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75ff29b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75f815d8a0 0x7f75f8147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75ff29b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f81484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75ff29b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8158930 con 0x7f75f815eb70 2024-01-19T04:57:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75ff29b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b42f0 con 0x7f75f815d8a0 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75ff29b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b5920 con 0x7f75f80cca00 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f81484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f81484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47718/0 (socket says 172.21.15.31:47718) 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 -- 172.21.15.31:0/3475187866 learned_addr learned my addr 172.21.15.31:0/3475187866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fc836640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75f815d8a0 0x7f75f8147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd037640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f80cca00 0x7f75f80b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 -- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f80cca00 msgr2=0x7f75f80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f80cca00 0x7f75f80b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 -- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75f815d8a0 msgr2=0x7f75f8147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75f815d8a0 0x7f75f8147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 -- 172.21.15.31:0/3475187866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0094d0 con 0x7f75f815eb70 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.013+0000 7f75fd838640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f81484b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f75e800d800 tx=0x7f75e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.014+0000 7f75f5ffb640 1 -- 172.21.15.31:0/3475187866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80096c0 con 0x7f75f815eb70 2024-01-19T04:57:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.014+0000 7f75f5ffb640 1 -- 172.21.15.31:0/3475187866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e8009840 con 0x7f75f815eb70 2024-01-19T04:57:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.014+0000 7f75f5ffb640 1 -- 172.21.15.31:0/3475187866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8009bf0 con 0x7f75f815eb70 2024-01-19T04:57:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.014+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f80b5c00 con 0x7f75f815eb70 2024-01-19T04:57:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.014+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f80b61c0 con 0x7f75f815eb70 2024-01-19T04:57:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.015+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f80b63b0 con 0x7f75f815eb70 2024-01-19T04:57:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.015+0000 7f75f5ffb640 1 -- 172.21.15.31:0/3475187866 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f75e8009d50 con 0x7f75f815eb70 2024-01-19T04:57:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.015+0000 7f75f5ffb640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f75e406aed0 0x7f75e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.016+0000 7f75f5ffb640 1 -- 172.21.15.31:0/3475187866 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75e8089a30 con 0x7f75f815eb70 2024-01-19T04:57:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.016+0000 7f75fd037640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f75e406aed0 0x7f75e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.016+0000 7f75fd037640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f75e406aed0 0x7f75e406d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009a60 tx=0x7f75f0003990 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.020+0000 7f75f5ffb640 1 -- 172.21.15.31:0/3475187866 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f75e8054900 con 0x7f75f815eb70 2024-01-19T04:57:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.197+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75f80696e0 con 0x7f75f815eb70 2024-01-19T04:57:14.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.198+0000 7f75f5ffb640 1 -- 172.21.15.31:0/3475187866 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f75e8018090 con 0x7f75f815eb70 2024-01-19T04:57:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:14.198 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":49,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:12.879709+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f75e406aed0 msgr2=0x7f75e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f75e406aed0 0x7f75e406d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009a60 tx=0x7f75f0003990 comp rx=0 tx=0).stop 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 msgr2=0x7f75f81484b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f81484b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f75e800d800 tx=0x7f75e800dcc0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 shutdown_connections 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f75e406aed0 0x7f75e406d390 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f815eb70 0x7f75f81484b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75f815d8a0 0x7f75f8147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 --2- 172.21.15.31:0/3475187866 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f75f80cca00 0x7f75f80b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 >> 172.21.15.31:0/3475187866 conn(0x7f75f806abe0 msgr2=0x7f75f80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:14.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 shutdown_connections 2024-01-19T04:57:14.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.200+0000 7f75ff29b640 1 -- 172.21.15.31:0/3475187866 wait complete. 2024-01-19T04:57:14.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-19T04:57:14.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.315+0000 7f7756382640 1 Processor -- start 2024-01-19T04:57:14.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.315+0000 7f7756382640 1 -- start start 2024-01-19T04:57:14.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f7756382640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f775015d700 0x7f77500d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f7756382640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77500d11b0 0x7f77500d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f7756382640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77500cef10 con 0x7f77500d1e10 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f7756382640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77500d2280 con 0x7f77500d11b0 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f7756382640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77500ce300 con 0x7f775015d700 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77500d11b0 0x7f77500d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77500d11b0 0x7f77500d18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47734/0 (socket says 172.21.15.31:47734) 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774f7fe640 1 -- 172.21.15.31:0/1827599490 learned_addr learned my addr 172.21.15.31:0/1827599490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774f7fe640 1 -- 172.21.15.31:0/1827599490 >> v2:172.21.15.137:3300/0 conn(0x7f775015d700 msgr2=0x7f77500d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774ffff640 1 --2- 172.21.15.31:0/1827599490 >> v2:172.21.15.137:3300/0 conn(0x7f775015d700 0x7f77500d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774f7fe640 1 --2- 172.21.15.31:0/1827599490 >> v2:172.21.15.137:3300/0 conn(0x7f775015d700 0x7f77500d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774f7fe640 1 -- 172.21.15.31:0/1827599490 >> v1:172.21.15.137:6789/0 conn(0x7f77500d1e10 legacy=0x7f7750161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774f7fe640 1 -- 172.21.15.31:0/1827599490 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77500ce5e0 con 0x7f77500d11b0 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.316+0000 7f774ffff640 1 --2- 172.21.15.31:0/1827599490 >> v2:172.21.15.137:3300/0 conn(0x7f775015d700 0x7f77500d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.317+0000 7f774f7fe640 1 --2- 172.21.15.31:0/1827599490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77500d18d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f77500b5e70 tx=0x7f774002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a298eaef4077a22 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.317+0000 7f774effd640 1 -- 172.21.15.31:0/1827599490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740030860 con 0x7f77500d11b0 2024-01-19T04:57:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.317+0000 7f774effd640 1 -- 172.21.15.31:0/1827599490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77400309c0 con 0x7f77500d11b0 2024-01-19T04:57:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.317+0000 7f774effd640 1 -- 172.21.15.31:0/1827599490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740030d50 con 0x7f77500d11b0 2024-01-19T04:57:14.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 -- 172.21.15.31:0/1827599490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 msgr2=0x7f77500d18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 --2- 172.21.15.31:0/1827599490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77500d18d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f77500b5e70 tx=0x7f774002b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 -- 172.21.15.31:0/1827599490 shutdown_connections 2024-01-19T04:57:14.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 --2- 172.21.15.31:0/1827599490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77500d18d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 --2- 172.21.15.31:0/1827599490 >> v2:172.21.15.137:3300/0 conn(0x7f775015d700 0x7f77500d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 -- 172.21.15.31:0/1827599490 >> 172.21.15.31:0/1827599490 conn(0x7f77500d32f0 msgr2=0x7f77500d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 -- 172.21.15.31:0/1827599490 shutdown_connections 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.318+0000 7f7756382640 1 -- 172.21.15.31:0/1827599490 wait complete. 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.319+0000 7f7756382640 1 Processor -- start 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.319+0000 7f7756382640 1 -- start start 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f7756382640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77501602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f7756382640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77500d1e10 0x7f7750160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f7756382640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f775015d700 0x7f77500cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f7756382640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750143540 con 0x7f77500d11b0 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f7756382640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750160d50 con 0x7f77500d1e10 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f7756382640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7750142ea0 con 0x7f775015d700 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77501602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77501602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47742/0 (socket says 172.21.15.31:47742) 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 -- 172.21.15.31:0/1994744276 learned_addr learned my addr 172.21.15.31:0/1994744276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774f7fe640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77500d1e10 0x7f7750160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f775491f640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f775015d700 0x7f77500cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 -- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f775015d700 msgr2=0x7f77500cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f775015d700 0x7f77500cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 -- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77500d1e10 msgr2=0x7f7750160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77500d1e10 0x7f7750160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 -- 172.21.15.31:0/1994744276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77400094d0 con 0x7f77500d11b0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774ffff640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77501602d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f774400c7f0 tx=0x7f774400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.320+0000 7f774cff9640 1 -- 172.21.15.31:0/1994744276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744007820 con 0x7f77500d11b0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.321+0000 7f774cff9640 1 -- 172.21.15.31:0/1994744276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7744007980 con 0x7f77500d11b0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.321+0000 7f774cff9640 1 -- 172.21.15.31:0/1994744276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744007cf0 con 0x7f77500d11b0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.321+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77500b6250 con 0x7f77500d11b0 2024-01-19T04:57:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.321+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7750143160 con 0x7f77500d11b0 2024-01-19T04:57:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.321+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77501437b0 con 0x7f77500d11b0 2024-01-19T04:57:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.322+0000 7f774cff9640 1 -- 172.21.15.31:0/1994744276 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f77440113f0 con 0x7f77500d11b0 2024-01-19T04:57:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.322+0000 7f774cff9640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f773c06aed0 0x7f773c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.323+0000 7f774cff9640 1 -- 172.21.15.31:0/1994744276 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77440898a0 con 0x7f77500d11b0 2024-01-19T04:57:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.323+0000 7f774f7fe640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f773c06aed0 0x7f773c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.323+0000 7f774f7fe640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f773c06aed0 0x7f773c06d390 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7740036000 tx=0x7f7740008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.327+0000 7f774cff9640 1 -- 172.21.15.31:0/1994744276 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7744054800 con 0x7f77500d11b0 2024-01-19T04:57:14.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:14.544+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7750074900 con 0x7f77500d11b0 2024-01-19T04:57:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.532+0000 7f774cff9640 1 -- 172.21.15.31:0/1994744276 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f774401f0b0 con 0x7f77500d11b0 2024-01-19T04:57:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f773c06aed0 msgr2=0x7f773c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f773c06aed0 0x7f773c06d390 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7740036000 tx=0x7f7740008030 comp rx=0 tx=0).stop 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 msgr2=0x7f77501602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77501602d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f774400c7f0 tx=0x7f774400ccb0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 shutdown_connections 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f773c06aed0 0x7f773c06d390 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f775015d700 0x7f77500cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77500d1e10 0x7f7750160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 --2- 172.21.15.31:0/1994744276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77500d11b0 0x7f77501602d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 >> 172.21.15.31:0/1994744276 conn(0x7f77500d32f0 msgr2=0x7f77500d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 shutdown_connections 2024-01-19T04:57:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.535+0000 7f7756382640 1 -- 172.21.15.31:0/1994744276 wait complete. 2024-01-19T04:57:15.548 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-19T04:57:15.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-19T04:57:15.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.658+0000 7fa217a3a640 1 Processor -- start 2024-01-19T04:57:15.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.659+0000 7fa217a3a640 1 -- start start 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.659+0000 7fa217a3a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa2100d1c20 0x7fa2100d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.659+0000 7fa217a3a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2100cf160 con 0x7fa21015d720 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.659+0000 7fa217a3a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2100ce8c0 con 0x7fa2100d3ea0 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.659+0000 7fa217a3a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2100ceba0 con 0x7fa2100d1c20 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa2157d6640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa21015d720 0x7fa2100d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60680/0 (socket says 172.21.15.31:60680) 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa2157d6640 1 -- 172.21.15.31:0/3703391764 learned_addr learned my addr 172.21.15.31:0/3703391764 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa214fd5640 1 --2- 172.21.15.31:0/3703391764 >> v2:172.21.15.137:3300/0 conn(0x7fa2100d1c20 0x7fa2100d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611399338 0 0) 0x7fa2100cf160 con 0x7fa21015d720 2024-01-19T04:57:15.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1fc003520 con 0x7fa21015d720 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691369690 0 0) 0x7fa1fc003520 con 0x7fa21015d720 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 >> v2:172.21.15.137:3300/0 conn(0x7fa2100d1c20 msgr2=0x7fa2100d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 --2- 172.21.15.31:0/3703391764 >> v2:172.21.15.137:3300/0 conn(0x7fa2100d1c20 0x7fa2100d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 >> v1:172.21.15.137:6789/0 conn(0x7fa2100d3ea0 legacy=0x7fa2100b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100cee80 con 0x7fa21015d720 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fa208001670 con 0x7fa21015d720 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 >> v1:172.21.15.31:6789/0 conn(0x7fa21015d720 legacy=0x7fa2100d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1fc003970 0x7fa1fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc0047c0 0x7fa1fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc0052a0 0x7fa1fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100ceba0 con 0x7fa1fc003970 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc00c170 con 0x7fa1fc0052a0 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc00c330 con 0x7fa1fc0047c0 2024-01-19T04:57:15.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa2157d6640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1fc003970 0x7fa1fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.660+0000 7fa215fd7640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc0052a0 0x7fa1fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa214fd5640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc0047c0 0x7fa1fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa2157d6640 1 -- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc0047c0 msgr2=0x7fa1fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa2157d6640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc0047c0 0x7fa1fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa2157d6640 1 -- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc0052a0 msgr2=0x7fa1fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa2157d6640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc0052a0 0x7fa1fc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa2157d6640 1 -- 172.21.15.31:0/3703391764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc003520 con 0x7fa1fc003970 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa215fd7640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc0052a0 0x7fa1fc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa215fd7640 1 -- 172.21.15.31:0/3703391764 reap_dead start 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa2157d6640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1fc003970 0x7fa1fc003d90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c210 tx=0x7fa1fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd5d9c424429aa9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.661+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208005ad0 con 0x7fa1fc003970 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208005c30 con 0x7fa1fc003970 2024-01-19T04:57:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa207fff640 1 -- 172.21.15.31:0/3703391764 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20802cad0 con 0x7fa1fc003970 2024-01-19T04:57:15.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa217a3a640 1 -- 172.21.15.31:0/3703391764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1fc003970 msgr2=0x7fa1fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:15.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa217a3a640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1fc003970 0x7fa1fc003d90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c210 tx=0x7fa1fc0057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa217a3a640 1 -- 172.21.15.31:0/3703391764 shutdown_connections 2024-01-19T04:57:15.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa217a3a640 1 --2- 172.21.15.31:0/3703391764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1fc003970 0x7fa1fc003d90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa217a3a640 1 -- 172.21.15.31:0/3703391764 >> 172.21.15.31:0/3703391764 conn(0x7fa2100d3310 msgr2=0x7fa2100d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:15.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa217a3a640 1 -- 172.21.15.31:0/3703391764 shutdown_connections 2024-01-19T04:57:15.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.662+0000 7fa217a3a640 1 -- 172.21.15.31:0/3703391764 wait complete. 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.663+0000 7fa217a3a640 1 Processor -- start 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.663+0000 7fa217a3a640 1 -- start start 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa217a3a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc003970 0x7fa2100c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa217a3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2100c9580 0x7fa21015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa217a3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2100b7f00 0x7fa2100b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa217a3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100b88c0 con 0x7fa2100b7f00 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa217a3a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100c7d60 con 0x7fa2100c9580 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa217a3a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100cd490 con 0x7fa1fc003970 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa215fd7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2100b7f00 0x7fa2100b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa215fd7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2100b7f00 0x7fa2100b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47756/0 (socket says 172.21.15.31:47756) 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa215fd7640 1 -- 172.21.15.31:0/1685423531 learned_addr learned my addr 172.21.15.31:0/1685423531 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa215fd7640 1 -- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc003970 msgr2=0x7fa2100c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.664+0000 7fa215fd7640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc003970 0x7fa2100c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa215fd7640 1 -- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2100c9580 msgr2=0x7fa21015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa215fd7640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2100c9580 0x7fa21015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa215fd7640 1 -- 172.21.15.31:0/1685423531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21015bce0 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa215fd7640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2100b7f00 0x7fa2100b8380 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa21008c490 tx=0x7fa20c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa205ffb640 1 -- 172.21.15.31:0/1685423531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00eed0 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa205ffb640 1 -- 172.21.15.31:0/1685423531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c003970 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2100c7fa0 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa205ffb640 1 -- 172.21.15.31:0/1685423531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c003d20 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.665+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2100c8560 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.666+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa210074900 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.667+0000 7fa205ffb640 1 -- 172.21.15.31:0/1685423531 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa20c0050d0 con 0x7fa2100b7f00 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.667+0000 7fa205ffb640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa1f406aed0 0x7fa1f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:15.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.667+0000 7fa205ffb640 1 -- 172.21.15.31:0/1685423531 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa20c089490 con 0x7fa2100b7f00 2024-01-19T04:57:15.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.667+0000 7fa2157d6640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa1f406aed0 0x7fa1f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:15.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.668+0000 7fa2157d6640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa1f406aed0 0x7fa1f406d390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa208034000 tx=0x7fa208039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:15.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.671+0000 7fa205ffb640 1 -- 172.21.15.31:0/1685423531 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa20c054230 con 0x7fa2100b7f00 2024-01-19T04:57:15.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:15.887+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa21015c030 con 0x7fa2100b7f00 2024-01-19T04:57:16.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.539+0000 7fa205ffb640 1 -- 172.21.15.31:0/1685423531 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa20c057850 con 0x7fa2100b7f00 2024-01-19T04:57:16.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa1f406aed0 msgr2=0x7fa1f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa1f406aed0 0x7fa1f406d390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa208034000 tx=0x7fa208039040 comp rx=0 tx=0).stop 2024-01-19T04:57:16.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2100b7f00 msgr2=0x7fa2100b8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2100b7f00 0x7fa2100b8380 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa21008c490 tx=0x7fa20c00eb90 comp rx=0 tx=0).stop 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 shutdown_connections 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa1f406aed0 0x7fa1f406d390 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2100b7f00 0x7fa2100b8380 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2100c9580 0x7fa21015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 --2- 172.21.15.31:0/1685423531 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa1fc003970 0x7fa2100c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 >> 172.21.15.31:0/1685423531 conn(0x7fa2100d3310 msgr2=0x7fa2100cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 shutdown_connections 2024-01-19T04:57:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.542+0000 7fa217a3a640 1 -- 172.21.15.31:0/1685423531 wait complete. 2024-01-19T04:57:16.554 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-19T04:57:16.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:57:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.661+0000 7fae88561640 1 Processor -- start 2024-01-19T04:57:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.661+0000 7fae88561640 1 -- start start 2024-01-19T04:57:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae88561640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fae800cc230 0x7fae800b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae88561640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae800b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae88561640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae80142890 con 0x7fae800c4db0 2024-01-19T04:57:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae88561640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae80142b70 con 0x7fae800cc230 2024-01-19T04:57:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae88561640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae80142e50 con 0x7fae80142320 2024-01-19T04:57:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae862fd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fae800cc230 0x7fae800b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae85afc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae800b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae85afc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae800b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:34018/0 (socket says 172.21.15.31:34018) 2024-01-19T04:57:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae85afc640 1 -- 172.21.15.31:0/2949596643 learned_addr learned my addr 172.21.15.31:0/2949596643 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.662+0000 7fae86afe640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fae800c4db0 0x7fae80143930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:33534/0 (socket says 172.21.15.31:33534) 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.663+0000 7fae85afc640 1 -- 172.21.15.31:0/2949596643 >> v2:172.21.15.137:3300/0 conn(0x7fae800cc230 msgr2=0x7fae800b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.663+0000 7fae85afc640 1 --2- 172.21.15.31:0/2949596643 >> v2:172.21.15.137:3300/0 conn(0x7fae800cc230 0x7fae800b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.663+0000 7fae85afc640 1 -- 172.21.15.31:0/2949596643 >> v1:172.21.15.137:6789/0 conn(0x7fae800c4db0 legacy=0x7fae80143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.663+0000 7fae85afc640 1 -- 172.21.15.31:0/2949596643 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80143130 con 0x7fae80142320 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.663+0000 7fae862fd640 1 --2- 172.21.15.31:0/2949596643 >> v2:172.21.15.137:3300/0 conn(0x7fae800cc230 0x7fae800b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.663+0000 7fae852fb640 1 -- 172.21.15.31:0/2949596643 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116551744 0 0) 0x7fae80142890 con 0x7fae800c4db0 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.663+0000 7fae85afc640 1 --2- 172.21.15.31:0/2949596643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae800b3280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae70009870 tx=0x7fae7002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b51d8462755aa17 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae852fb640 1 -- 172.21.15.31:0/2949596643 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70030840 con 0x7fae80142320 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae852fb640 1 -- 172.21.15.31:0/2949596643 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae700309a0 con 0x7fae80142320 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae852fb640 1 -- 172.21.15.31:0/2949596643 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70030d30 con 0x7fae80142320 2024-01-19T04:57:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 -- 172.21.15.31:0/2949596643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 msgr2=0x7fae800b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 --2- 172.21.15.31:0/2949596643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae800b3280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae70009870 tx=0x7fae7002b980 comp rx=0 tx=0).stop 2024-01-19T04:57:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 -- 172.21.15.31:0/2949596643 shutdown_connections 2024-01-19T04:57:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 --2- 172.21.15.31:0/2949596643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae800b3280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 --2- 172.21.15.31:0/2949596643 >> v2:172.21.15.137:3300/0 conn(0x7fae800cc230 0x7fae800b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 -- 172.21.15.31:0/2949596643 >> 172.21.15.31:0/2949596643 conn(0x7fae8006abe0 msgr2=0x7fae80066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 -- 172.21.15.31:0/2949596643 shutdown_connections 2024-01-19T04:57:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.664+0000 7fae88561640 1 -- 172.21.15.31:0/2949596643 wait complete. 2024-01-19T04:57:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.665+0000 7fae88561640 1 Processor -- start 2024-01-19T04:57:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.665+0000 7fae88561640 1 -- start start 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae88561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae800c4db0 0x7fae8015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae88561640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae800cc230 0x7fae8006d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae88561640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae801e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae88561640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8015b1a0 con 0x7fae800c4db0 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae88561640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8006b6c0 con 0x7fae800cc230 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae88561640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8006c240 con 0x7fae80142320 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae862fd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae800c4db0 0x7fae8015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae862fd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae800c4db0 0x7fae8015ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47766/0 (socket says 172.21.15.31:47766) 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae862fd640 1 -- 172.21.15.31:0/1052623790 learned_addr learned my addr 172.21.15.31:0/1052623790 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae86afe640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae801e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.666+0000 7fae85afc640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae800cc230 0x7fae8006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae862fd640 1 -- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 msgr2=0x7fae801e82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae862fd640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae801e82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae862fd640 1 -- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae800cc230 msgr2=0x7fae8006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae862fd640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae800cc230 0x7fae8006d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae862fd640 1 -- 172.21.15.31:0/1052623790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c009400 con 0x7fae800c4db0 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae85afc640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae800cc230 0x7fae8006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae862fd640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae800c4db0 0x7fae8015ac60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00ebe0 tx=0x7fae7c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae76ffd640 1 -- 172.21.15.31:0/1052623790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009760 con 0x7fae800c4db0 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae76ffd640 1 -- 172.21.15.31:0/1052623790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c00cc30 con 0x7fae800c4db0 2024-01-19T04:57:16.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae700094d0 con 0x7fae800c4db0 2024-01-19T04:57:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae76ffd640 1 -- 172.21.15.31:0/1052623790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00eee0 con 0x7fae800c4db0 2024-01-19T04:57:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.667+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8006b910 con 0x7fae800c4db0 2024-01-19T04:57:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.668+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae800696e0 con 0x7fae800c4db0 2024-01-19T04:57:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.668+0000 7fae76ffd640 1 -- 172.21.15.31:0/1052623790 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fae7c013470 con 0x7fae800c4db0 2024-01-19T04:57:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.669+0000 7fae76ffd640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fae6406aed0 0x7fae6406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.669+0000 7fae76ffd640 1 -- 172.21.15.31:0/1052623790 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae7c0898d0 con 0x7fae800c4db0 2024-01-19T04:57:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.669+0000 7fae85afc640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fae6406aed0 0x7fae6406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.670+0000 7fae85afc640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fae6406aed0 0x7fae6406d390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae70036000 tx=0x7fae7003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.673+0000 7fae76ffd640 1 -- 172.21.15.31:0/1052623790 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fae7c054850 con 0x7fae800c4db0 2024-01-19T04:57:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.888+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae80141fc0 con 0x7fae800c4db0 2024-01-19T04:57:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.889+0000 7fae76ffd640 1 -- 172.21.15.31:0/1052623790 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fae7c057e70 con 0x7fae800c4db0 2024-01-19T04:57:16.888 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:16.888 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/3002045021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3002045021},{"type":"v1","addr":"172.21.15.137:6836","nonce":3002045021}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1415057458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":1415057458},{"type":"v1","addr":"172.21.15.137:6837","nonce":1415057458}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5919,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2062734261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2062734261},{"type":"v1","addr":"172.21.15.31:6833","nonce":2062734261}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:57:12.880027+0000","modified":"2024-01-19T04:57:15.888893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1600502506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1600502506},{"type":"v1","addr":"172.21.15.31:6835","nonce":1600502506}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-19T04:57:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-01-19T04:57:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fae6406aed0 msgr2=0x7fae6406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fae6406aed0 0x7fae6406d390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae70036000 tx=0x7fae7003b040 comp rx=0 tx=0).stop 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae800c4db0 msgr2=0x7fae8015ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae800c4db0 0x7fae8015ac60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00ebe0 tx=0x7fae7c00c520 comp rx=0 tx=0).stop 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 shutdown_connections 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fae6406aed0 0x7fae6406d390 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fae80142320 0x7fae801e82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae800cc230 0x7fae8006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 --2- 172.21.15.31:0/1052623790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae800c4db0 0x7fae8015ac60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 >> 172.21.15.31:0/1052623790 conn(0x7fae8006abe0 msgr2=0x7fae800c5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 shutdown_connections 2024-01-19T04:57:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:16.891+0000 7fae88561640 1 -- 172.21.15.31:0/1052623790 wait complete. 2024-01-19T04:57:16.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.004+0000 7f096d45d640 1 Processor -- start 2024-01-19T04:57:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.004+0000 7f096d45d640 1 -- start start 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.004+0000 7f096d45d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096806d6b0 0x7f09680b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.004+0000 7f096d45d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f096815b260 0x7f096806da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.004+0000 7f096d45d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09680b2ee0 con 0x7f096813f9a0 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.004+0000 7f096d45d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680cba60 con 0x7f096815b260 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.004+0000 7f096d45d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096806dfd0 con 0x7f096806d6b0 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09667fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f096815b260 0x7f096806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09667fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f096815b260 0x7f096806da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47780/0 (socket says 172.21.15.31:47780) 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09677fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f096813f9a0 0x7f096813fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60684/0 (socket says 172.21.15.31:60684) 2024-01-19T04:57:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09667fc640 1 -- 172.21.15.31:0/4046058018 learned_addr learned my addr 172.21.15.31:0/4046058018 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f0966ffd640 1 --2- 172.21.15.31:0/4046058018 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096806d6b0 0x7f09680b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09667fc640 1 -- 172.21.15.31:0/4046058018 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096806d6b0 msgr2=0x7f09680b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09667fc640 1 --2- 172.21.15.31:0/4046058018 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096806d6b0 0x7f09680b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09667fc640 1 -- 172.21.15.31:0/4046058018 >> v1:172.21.15.31:6789/0 conn(0x7f096813f9a0 legacy=0x7f096813fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f09667fc640 1 -- 172.21.15.31:0/4046058018 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096815b6c0 con 0x7f096815b260 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f0965ffb640 1 -- 172.21.15.31:0/4046058018 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762397677 0 0) 0x7f09680b2ee0 con 0x7f096813f9a0 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.005+0000 7f0966ffd640 1 --2- 172.21.15.31:0/4046058018 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096806d6b0 0x7f09680b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f09667fc640 1 --2- 172.21.15.31:0/4046058018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815b260 0x7f096806da90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0950009870 tx=0x7f095002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979eb4920a494f45 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f0965ffb640 1 -- 172.21.15.31:0/4046058018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950031640 con 0x7f096815b260 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f0965ffb640 1 -- 172.21.15.31:0/4046058018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09500317a0 con 0x7f096815b260 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f0965ffb640 1 -- 172.21.15.31:0/4046058018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950031b30 con 0x7f096815b260 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 -- 172.21.15.31:0/4046058018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815b260 msgr2=0x7f096806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 --2- 172.21.15.31:0/4046058018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815b260 0x7f096806da90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0950009870 tx=0x7f095002b8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 -- 172.21.15.31:0/4046058018 shutdown_connections 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 --2- 172.21.15.31:0/4046058018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815b260 0x7f096806da90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 --2- 172.21.15.31:0/4046058018 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096806d6b0 0x7f09680b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 -- 172.21.15.31:0/4046058018 >> 172.21.15.31:0/4046058018 conn(0x7f096806a790 msgr2=0x7f096806aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:17.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 -- 172.21.15.31:0/4046058018 shutdown_connections 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.006+0000 7f096d45d640 1 -- 172.21.15.31:0/4046058018 wait complete. 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 Processor -- start 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 -- start start 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f096806d6b0 0x7f0968152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096813f9a0 0x7f09680cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096815b260 0x7f09681595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09681f4370 con 0x7f096813f9a0 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0968152850 con 0x7f096806d6b0 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f096d45d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680cda50 con 0x7f096815b260 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f09667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096813f9a0 0x7f09680cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f09667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096813f9a0 0x7f09680cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47794/0 (socket says 172.21.15.31:47794) 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.008+0000 7f09667fc640 1 -- 172.21.15.31:0/94213519 learned_addr learned my addr 172.21.15.31:0/94213519 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09677fe640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096815b260 0x7f09681595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f0966ffd640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f096806d6b0 0x7f0968152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09667fc640 1 -- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096815b260 msgr2=0x7f09681595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09667fc640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096815b260 0x7f09681595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09667fc640 1 -- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f096806d6b0 msgr2=0x7f0968152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09667fc640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f096806d6b0 0x7f0968152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09667fc640 1 -- 172.21.15.31:0/94213519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500094d0 con 0x7f096813f9a0 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f0966ffd640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f096806d6b0 0x7f0968152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09677fe640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096815b260 0x7f09681595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f09667fc640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096813f9a0 0x7f09680cd090 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f095002b930 tx=0x7f09500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f095f7fe640 1 -- 172.21.15.31:0/94213519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095003ac60 con 0x7f096813f9a0 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f095f7fe640 1 -- 172.21.15.31:0/94213519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095003ae00 con 0x7f096813f9a0 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09680cdd30 con 0x7f096813f9a0 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f095f7fe640 1 -- 172.21.15.31:0/94213519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950044630 con 0x7f096813f9a0 2024-01-19T04:57:17.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.009+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09680ce2f0 con 0x7f096813f9a0 2024-01-19T04:57:17.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.010+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09680d0080 con 0x7f096813f9a0 2024-01-19T04:57:17.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.011+0000 7f095f7fe640 1 -- 172.21.15.31:0/94213519 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f095002f050 con 0x7f096813f9a0 2024-01-19T04:57:17.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.011+0000 7f095f7fe640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f094806af00 0x7f094806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.011+0000 7f095f7fe640 1 -- 172.21.15.31:0/94213519 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09500ab640 con 0x7f096813f9a0 2024-01-19T04:57:17.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.011+0000 7f0966ffd640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f094806af00 0x7f094806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.012+0000 7f0966ffd640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f094806af00 0x7f094806d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f095800e550 tx=0x7f0958007ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.015+0000 7f095f7fe640 1 -- 172.21.15.31:0/94213519 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0950076510 con 0x7f096813f9a0 2024-01-19T04:57:17.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.195+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f096806cd60 con 0x7f096813f9a0 2024-01-19T04:57:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.196+0000 7f095f7fe640 1 -- 172.21.15.31:0/94213519 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f0950079b30 con 0x7f096813f9a0 2024-01-19T04:57:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:17.195 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:17.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.197+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f094806af00 msgr2=0x7f094806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.197+0000 7f096d45d640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f094806af00 0x7f094806d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f095800e550 tx=0x7f0958007ab0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.197+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096813f9a0 msgr2=0x7f09680cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.197+0000 7f096d45d640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096813f9a0 0x7f09680cd090 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f095002b930 tx=0x7f09500046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 shutdown_connections 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f094806af00 0x7f094806d3c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f096815b260 0x7f09681595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096813f9a0 0x7f09680cd090 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 --2- 172.21.15.31:0/94213519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f096806d6b0 0x7f0968152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 >> 172.21.15.31:0/94213519 conn(0x7f096806a790 msgr2=0x7f096806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 shutdown_connections 2024-01-19T04:57:17.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.198+0000 7f096d45d640 1 -- 172.21.15.31:0/94213519 wait complete. 2024-01-19T04:57:17.208 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-19T04:57:17.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:57:17.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.310+0000 7f5ffa1de640 1 Processor -- start 2024-01-19T04:57:17.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.310+0000 7f5ffa1de640 1 -- start start 2024-01-19T04:57:17.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5ffa1de640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5ff40b1f40 0x7f5ff415b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5ffa1de640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff40b2ee0 con 0x7f5ff406d6b0 2024-01-19T04:57:17.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5ffa1de640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff40cba60 con 0x7f5ff413df90 2024-01-19T04:57:17.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5ffa1de640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff415b7a0 con 0x7f5ff40b1f40 2024-01-19T04:57:17.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5feaffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5ff40b1f40 0x7f5ff415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5ff37fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5ff406d6b0 0x7f5ff406da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60688/0 (socket says 172.21.15.31:60688) 2024-01-19T04:57:17.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5ff37fe640 1 -- 172.21.15.31:0/1455151653 learned_addr learned my addr 172.21.15.31:0/1455151653 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:17.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5feaffd640 1 -- 172.21.15.31:0/1455151653 >> v1:172.21.15.137:6789/0 conn(0x7f5ff413df90 legacy=0x7f5ff413e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:17.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5feaffd640 1 -- 172.21.15.31:0/1455151653 >> v1:172.21.15.31:6789/0 conn(0x7f5ff406d6b0 legacy=0x7f5ff406da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.312+0000 7f5feaffd640 1 -- 172.21.15.31:0/1455151653 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40b23d0 con 0x7f5ff40b1f40 2024-01-19T04:57:17.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.311+0000 7f5ff2ffd640 1 -- 172.21.15.31:0/1455151653 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393787098 0 0) 0x7f5ff40b2ee0 con 0x7f5ff406d6b0 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.312+0000 7f5feaffd640 1 --2- 172.21.15.31:0/1455151653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff40b1f40 0x7f5ff415b260 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097a0 tx=0x7f5fdc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945daa29d12911df server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.312+0000 7f5ff2ffd640 1 -- 172.21.15.31:0/1455151653 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc02e070 con 0x7f5ff40b1f40 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.312+0000 7f5ff2ffd640 1 -- 172.21.15.31:0/1455151653 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc0315a0 con 0x7f5ff40b1f40 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.312+0000 7f5ff2ffd640 1 -- 172.21.15.31:0/1455151653 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc031930 con 0x7f5ff40b1f40 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.313+0000 7f5ffa1de640 1 -- 172.21.15.31:0/1455151653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff40b1f40 msgr2=0x7f5ff415b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.313+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/1455151653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff40b1f40 0x7f5ff415b260 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097a0 tx=0x7f5fdc02b840 comp rx=0 tx=0).stop 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.313+0000 7f5ffa1de640 1 -- 172.21.15.31:0/1455151653 shutdown_connections 2024-01-19T04:57:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.313+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/1455151653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff40b1f40 0x7f5ff415b260 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.313+0000 7f5ffa1de640 1 -- 172.21.15.31:0/1455151653 >> 172.21.15.31:0/1455151653 conn(0x7f5ff406a790 msgr2=0x7f5ff406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:17.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.313+0000 7f5ffa1de640 1 -- 172.21.15.31:0/1455151653 shutdown_connections 2024-01-19T04:57:17.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.313+0000 7f5ffa1de640 1 -- 172.21.15.31:0/1455151653 wait complete. 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.314+0000 7f5ffa1de640 1 Processor -- start 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.314+0000 7f5ffa1de640 1 -- start start 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ffa1de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff406d6b0 0x7f5ff41e1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ffa1de640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ff40b1f40 0x7f5ff4130900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ffa1de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ff413df90 0x7f5ff4130e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ffa1de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff4143570 con 0x7f5ff406d6b0 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ffa1de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff4143990 con 0x7f5ff413df90 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ffa1de640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41428f0 con 0x7f5ff40b1f40 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff406d6b0 0x7f5ff41e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff406d6b0 0x7f5ff41e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47814/0 (socket says 172.21.15.31:47814) 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 -- 172.21.15.31:0/635744606 learned_addr learned my addr 172.21.15.31:0/635744606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 -- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ff40b1f40 msgr2=0x7f5ff4130900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5feaffd640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ff40b1f40 0x7f5ff4130900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ff40b1f40 0x7f5ff4130900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 -- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ff413df90 msgr2=0x7f5ff4130e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff3fff640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ff413df90 0x7f5ff4130e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ff413df90 0x7f5ff4130e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.315+0000 7f5ff37fe640 1 -- 172.21.15.31:0/635744606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc009400 con 0x7f5ff406d6b0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.316+0000 7f5ff37fe640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff406d6b0 0x7f5ff41e1180 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c4d0 tx=0x7f5fe400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.316+0000 7f5ff0ff9640 1 -- 172.21.15.31:0/635744606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40074e0 con 0x7f5ff406d6b0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.316+0000 7f5ff0ff9640 1 -- 172.21.15.31:0/635744606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4007660 con 0x7f5ff406d6b0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.316+0000 7f5ff0ff9640 1 -- 172.21.15.31:0/635744606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4004a20 con 0x7f5ff406d6b0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.316+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff4142bd0 con 0x7f5ff406d6b0 2024-01-19T04:57:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.316+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff4143190 con 0x7f5ff406d6b0 2024-01-19T04:57:17.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.317+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff40696e0 con 0x7f5ff406d6b0 2024-01-19T04:57:17.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.317+0000 7f5ff0ff9640 1 -- 172.21.15.31:0/635744606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5fe4004b80 con 0x7f5ff406d6b0 2024-01-19T04:57:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.318+0000 7f5ff0ff9640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5fd406aed0 0x7f5fd406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.318+0000 7f5ff0ff9640 1 -- 172.21.15.31:0/635744606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fe4088b90 con 0x7f5ff406d6b0 2024-01-19T04:57:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.318+0000 7f5feaffd640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5fd406aed0 0x7f5fd406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.318+0000 7f5feaffd640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5fd406aed0 0x7f5fd406d390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc036000 tx=0x7f5fdc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.322+0000 7f5ff0ff9640 1 -- 172.21.15.31:0/635744606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5fe4053af0 con 0x7f5ff406d6b0 2024-01-19T04:57:17.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.496+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ff40b2920 con 0x7f5fd406aed0 2024-01-19T04:57:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.504+0000 7f5ff0ff9640 1 -- 172.21.15.31:0/635744606 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277783 (secure 0 0 0) 0x7f5fdc0319b0 con 0x7f5fd406aed0 2024-01-19T04:57:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:57:17.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5fd406aed0 msgr2=0x7f5fd406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5fd406aed0 0x7f5fd406d390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc036000 tx=0x7f5fdc008030 comp rx=0 tx=0).stop 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff406d6b0 msgr2=0x7f5ff41e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff406d6b0 0x7f5ff41e1180 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c4d0 tx=0x7f5fe400c990 comp rx=0 tx=0).stop 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 shutdown_connections 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5fd406aed0 0x7f5fd406d390 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ff413df90 0x7f5ff4130e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5ff40b1f40 0x7f5ff4130900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 --2- 172.21.15.31:0/635744606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff406d6b0 0x7f5ff41e1180 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 >> 172.21.15.31:0/635744606 conn(0x7f5ff406a790 msgr2=0x7f5ff413f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 shutdown_connections 2024-01-19T04:57:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.507+0000 7f5ffa1de640 1 -- 172.21.15.31:0/635744606 wait complete. 2024-01-19T04:57:17.517 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-01-19T04:57:15.588519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221656,"kb_used_data":5416,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753064,"statfs":{"total":773094113280,"available":772867137536,"internally_reserved":0,"allocated":5545984,"data_stored":2957176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.078560"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.515284+0000","last_change":"2024-01-19T04:57:11.515284+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.515284+0000","last_undegraded":"2024-01-19T04:57:11.515284+0000","last_fullsized":"2024-01-19T04:57:11.515284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514545+0000","last_change":"2024-01-19T04:57:11.514545+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514545+0000","last_undegraded":"2024-01-19T04:57:11.514545+0000","last_fullsized":"2024-01-19T04:57:11.514545+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514479+0000","last_change":"2024-01-19T04:57:11.514479+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514479+0000","last_undegraded":"2024-01-19T04:57:11.514479+0000","last_fullsized":"2024-01-19T04:57:11.514479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.515294+0000","last_change":"2024-01-19T04:57:11.515294+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.515294+0000","last_undegraded":"2024-01-19T04:57:11.515294+0000","last_fullsized":"2024-01-19T04:57:11.515294+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.516073+0000","last_change":"2024-01-19T04:57:11.516073+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.516073+0000","last_undegraded":"2024-01-19T04:57:11.516073+0000","last_fullsized":"2024-01-19T04:57:11.516073+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512159+0000","last_change":"2024-01-19T04:57:11.512159+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512159+0000","last_undegraded":"2024-01-19T04:57:11.512159+0000","last_fullsized":"2024-01-19T04:57:11.512159+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514593+0000","last_change":"2024-01-19T04:57:11.514593+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514593+0000","last_undegraded":"2024-01-19T04:57:11.514593+0000","last_fullsized":"2024-01-19T04:57:11.514593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514737+0000","last_change":"2024-01-19T04:57:11.514737+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514737+0000","last_undegraded":"2024-01-19T04:57:11.514737+0000","last_fullsized":"2024-01-19T04:57:11.514737+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512336+0000","last_change":"2024-01-19T04:57:11.512336+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512336+0000","last_undegraded":"2024-01-19T04:57:11.512336+0000","last_fullsized":"2024-01-19T04:57:11.512336+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512165+0000","last_change":"2024-01-19T04:57:11.512165+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512165+0000","last_undegraded":"2024-01-19T04:57:11.512165+0000","last_fullsized":"2024-01-19T04:57:11.512165+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512826+0000","last_change":"2024-01-19T04:57:11.512826+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512826+0000","last_undegraded":"2024-01-19T04:57:11.512826+0000","last_fullsized":"2024-01-19T04:57:11.512826+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.513104+0000","last_change":"2024-01-19T04:57:11.513104+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.513104+0000","last_undegraded":"2024-01-19T04:57:11.513104+0000","last_fullsized":"2024-01-19T04:57:11.513104+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.515531+0000","last_change":"2024-01-19T04:57:11.515531+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.515531+0000","last_undegraded":"2024-01-19T04:57:11.515531+0000","last_fullsized":"2024-01-19T04:57:11.515531+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.515567+0000","last_change":"2024-01-19T04:57:11.515567+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.515567+0000","last_undegraded":"2024-01-19T04:57:11.515567+0000","last_fullsized":"2024-01-19T04:57:11.515567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512807+0000","last_change":"2024-01-19T04:57:11.512807+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512807+0000","last_undegraded":"2024-01-19T04:57:11.512807+0000","last_fullsized":"2024-01-19T04:57:11.512807+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.513426+0000","last_change":"2024-01-19T04:57:11.513426+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.513426+0000","last_undegraded":"2024-01-19T04:57:11.513426+0000","last_fullsized":"2024-01-19T04:57:11.513426+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.515873+0000","last_change":"2024-01-19T04:57:11.515873+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.515873+0000","last_undegraded":"2024-01-19T04:57:11.515873+0000","last_fullsized":"2024-01-19T04:57:11.515873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514818+0000","last_change":"2024-01-19T04:57:11.514818+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514818+0000","last_undegraded":"2024-01-19T04:57:11.514818+0000","last_fullsized":"2024-01-19T04:57:11.514818+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514792+0000","last_change":"2024-01-19T04:57:11.514792+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514792+0000","last_undegraded":"2024-01-19T04:57:11.514792+0000","last_fullsized":"2024-01-19T04:57:11.514792+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514593+0000","last_change":"2024-01-19T04:57:11.514593+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514593+0000","last_undegraded":"2024-01-19T04:57:11.514593+0000","last_fullsized":"2024-01-19T04:57:11.514593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.516004+0000","last_change":"2024-01-19T04:57:11.516004+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.516004+0000","last_undegraded":"2024-01-19T04:57:11.516004+0000","last_fullsized":"2024-01-19T04:57:11.516004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.513027+0000","last_change":"2024-01-19T04:57:11.513027+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.513027+0000","last_undegraded":"2024-01-19T04:57:11.513027+0000","last_fullsized":"2024-01-19T04:57:11.513027+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.515516+0000","last_change":"2024-01-19T04:57:11.515516+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.515516+0000","last_undegraded":"2024-01-19T04:57:11.515516+0000","last_fullsized":"2024-01-19T04:57:11.515516+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514586+0000","last_change":"2024-01-19T04:57:11.514586+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514586+0000","last_undegraded":"2024-01-19T04:57:11.514586+0000","last_fullsized":"2024-01-19T04:57:11.514586+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514605+0000","last_change":"2024-01-19T04:57:11.514605+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514605+0000","last_undegraded":"2024-01-19T04:57:11.514605+0000","last_fullsized":"2024-01-19T04:57:11.514605+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512303+0000","last_change":"2024-01-19T04:57:11.512303+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512303+0000","last_undegraded":"2024-01-19T04:57:11.512303+0000","last_fullsized":"2024-01-19T04:57:11.512303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514807+0000","last_change":"2024-01-19T04:57:11.514807+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514807+0000","last_undegraded":"2024-01-19T04:57:11.514807+0000","last_fullsized":"2024-01-19T04:57:11.514807+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512388+0000","last_change":"2024-01-19T04:57:11.512388+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512388+0000","last_undegraded":"2024-01-19T04:57:11.512388+0000","last_fullsized":"2024-01-19T04:57:11.512388+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512398+0000","last_change":"2024-01-19T04:57:11.512398+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512398+0000","last_undegraded":"2024-01-19T04:57:11.512398+0000","last_fullsized":"2024-01-19T04:57:11.512398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.512170+0000","last_change":"2024-01-19T04:57:11.512170+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.512170+0000","last_undegraded":"2024-01-19T04:57:11.512170+0000","last_fullsized":"2024-01-19T04:57:11.512170+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.514782+0000","last_change":"2024-01-19T04:57:11.514782+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.514782+0000","last_undegraded":"2024-01-19T04:57:11.514782+0000","last_fullsized":"2024-01-19T04:57:11.514782+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-19T04:57:12.509740+0000","last_change":"2024-01-19T04:57:12.509740+0000","last_active":"2024-01-19T04:57:12.509740+0000","last_peered":"2024-01-19T04:57:12.509740+0000","last_clean":"2024-01-19T04:57:12.509740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:12.509740+0000","last_undegraded":"2024-01-19T04:57:12.509740+0000","last_fullsized":"2024-01-19T04:57:12.509740+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-01-19T04:57:11.515280+0000","last_change":"2024-01-19T04:57:11.515280+0000","last_active":"2024-01-19T04:57:11.507731+0000","last_peered":"2024-01-19T04:57:11.507731+0000","last_clean":"2024-01-19T04:57:11.507731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-19T04:57:11.515280+0000","last_undegraded":"2024-01-19T04:57:11.515280+0000","last_fullsized":"2024-01-19T04:57:11.515280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-19T04:57:11.512142+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:57:11.512142+0000","last_peered":"2024-01-19T04:57:11.512142+0000","last_clean":"2024-01-19T04:57:11.512142+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:57:11.512142+0000","last_undegraded":"2024-01-19T04:57:11.512142+0000","last_fullsized":"2024-01-19T04:57:11.512142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1138688,"data_stored":812423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73099999999999998,"5min":0.753,"15min":0.753},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68200000000000005,"5min":0.745,"15min":0.745},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68500000000000005,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.621},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.504},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":222055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.312,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.69999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46000000000000002}]},{"osd":1,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.59899999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68799999999999994,"5min":0.876,"15min":0.876},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.69499999999999995,"5min":0.754,"15min":0.754},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.877,"15min":0.877},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.74099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":544768,"data_stored":222055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.377},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92800000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.504,"15min":0.505},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":544768,"data_stored":222055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98299999999999998,"5min":1.103,"15min":1.103},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84799999999999998,"5min":1.2,"15min":1.2},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0049999999999999,"5min":1.321,"15min":1.321},"last":0.441},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":1.046,"15min":1.046},"last":0.501}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.48499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0309999999999999,"5min":1.258,"15min":1.258},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":222055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.436},{"interface":"front","average":{"1min":0.318,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30599999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53200000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.88,"15min":0.88},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.755,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80300000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71299999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.629},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81999999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":544768,"data_stored":222055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67700000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.26600000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":1.47,"15min":1.47},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.318},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":1.006,"15min":1.006},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.752},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90900000000000003,"5min":1.355,"15min":1.355},"last":0.82299999999999995}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67700000000000005}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82999999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81899999999999995,"5min":0.995,"15min":0.995},"last":0.66500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":544768,"data_stored":222055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56299999999999994,"5min":1.397,"15min":1.397},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.23799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":1.423,"15min":1.423},"last":0.30599999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66700000000000004,"5min":1.446,"15min":1.446},"last":0.375},{"interface":"front","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.432},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.27600000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":1.619,"15min":1.619},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":1.585,"15min":1.585},"last":0.505},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":1.333,"15min":1.333},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.55500000000000005}]},{"osd":7,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83899999999999997,"5min":1.367,"15min":1.367},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.624},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1138688,"data_stored":812423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.36899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46400000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.29799999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.42299999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.373}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81200000000000006,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44500000000000001}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.044,"15min":1.044},"last":0.52300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:57:17.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:57:17.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.660+0000 7f71821bb640 1 Processor -- start 2024-01-19T04:57:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.660+0000 7f71821bb640 1 -- start start 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f71821bb640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f717c15d8a0 0x7f717c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f71821bb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c06bf30 con 0x7f717c15eb70 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f71821bb640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c0cb9e0 con 0x7f717c15f0c0 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f71821bb640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c0cbcc0 con 0x7f717c15d8a0 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717affd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f717c15eb70 0x7f717c0cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60704/0 (socket says 172.21.15.31:60704) 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717affd640 1 -- 172.21.15.31:0/2131604099 learned_addr learned my addr 172.21.15.31:0/2131604099 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717b7fe640 1 --2- 172.21.15.31:0/2131604099 >> v2:172.21.15.137:3300/0 conn(0x7f717c15d8a0 0x7f717c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574045923 0 0) 0x7f717c06bf30 con 0x7f717c15eb70 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7164003520 con 0x7f717c15eb70 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1531194435 0 0) 0x7f7164003520 con 0x7f717c15eb70 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 >> v2:172.21.15.137:3300/0 conn(0x7f717c15d8a0 msgr2=0x7f717c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 --2- 172.21.15.31:0/2131604099 >> v2:172.21.15.137:3300/0 conn(0x7f717c15d8a0 0x7f717c06b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 >> v1:172.21.15.137:6789/0 conn(0x7f717c15f0c0 legacy=0x7f717c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0cbfa0 con 0x7f717c15eb70 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f716c001670 con 0x7f717c15eb70 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 >> v1:172.21.15.31:6789/0 conn(0x7f717c15eb70 legacy=0x7f717c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7164003970 0x7f7164003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71640047c0 0x7f7164004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7164005180 0x7f716400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0cbcc0 con 0x7f7164003970 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716400c120 con 0x7f7164005180 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.661+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716400c2e0 con 0x7f71640047c0 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717b7fe640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7164003970 0x7f7164003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717bfff640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7164005180 0x7f716400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717affd640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71640047c0 0x7f7164004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717b7fe640 1 -- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71640047c0 msgr2=0x7f7164004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717b7fe640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71640047c0 0x7f7164004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717b7fe640 1 -- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7164005180 msgr2=0x7f716400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717b7fe640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7164005180 0x7f716400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717b7fe640 1 -- 172.21.15.31:0/2131604099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164003520 con 0x7f7164003970 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717bfff640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7164005180 0x7f716400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.662+0000 7f717bfff640 1 -- 172.21.15.31:0/2131604099 reap_dead start 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f717b7fe640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7164003970 0x7f7164003d90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7170009ac0 tx=0x7f71640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1637fd331658ff server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717002c6b0 con 0x7f7164003970 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717002c810 con 0x7f7164003970 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f717a7fc640 1 -- 172.21.15.31:0/2131604099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717002cba0 con 0x7f7164003970 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f71821bb640 1 -- 172.21.15.31:0/2131604099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7164003970 msgr2=0x7f7164003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f71821bb640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7164003970 0x7f7164003d90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7170009ac0 tx=0x7f71640057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f71821bb640 1 -- 172.21.15.31:0/2131604099 shutdown_connections 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f71821bb640 1 --2- 172.21.15.31:0/2131604099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7164003970 0x7f7164003d90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f71821bb640 1 -- 172.21.15.31:0/2131604099 >> 172.21.15.31:0/2131604099 conn(0x7f717c06abe0 msgr2=0x7f717c066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f71821bb640 1 -- 172.21.15.31:0/2131604099 shutdown_connections 2024-01-19T04:57:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.663+0000 7f71821bb640 1 -- 172.21.15.31:0/2131604099 wait complete. 2024-01-19T04:57:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.664+0000 7f71821bb640 1 Processor -- start 2024-01-19T04:57:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f71821bb640 1 -- start start 2024-01-19T04:57:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f71821bb640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7164003970 0x7f717c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f71821bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f717c154780 0x7f717c0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f71821bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f717c0d0b20 0x7f717c0d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f71821bb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0d14f0 con 0x7f717c0d0b20 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f71821bb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0b46b0 con 0x7f717c154780 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f71821bb640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0b4990 con 0x7f7164003970 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f717bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f717c0d0b20 0x7f717c0d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f717bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f717c0d0b20 0x7f717c0d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47842/0 (socket says 172.21.15.31:47842) 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.665+0000 7f717bfff640 1 -- 172.21.15.31:0/1035912518 learned_addr learned my addr 172.21.15.31:0/1035912518 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717b7fe640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7164003970 0x7f717c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717bfff640 1 -- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7164003970 msgr2=0x7f717c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717bfff640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7164003970 0x7f717c0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717affd640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f717c154780 0x7f717c0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717bfff640 1 -- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f717c154780 msgr2=0x7f717c0b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717bfff640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f717c154780 0x7f717c0b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717bfff640 1 -- 172.21.15.31:0/1035912518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0b4c70 con 0x7f717c0d0b20 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717b7fe640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7164003970 0x7f717c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717affd640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f717c154780 0x7f717c0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f717bfff640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f717c0d0b20 0x7f717c0d0fb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f717c162590 tx=0x7f717400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f7163fff640 1 -- 172.21.15.31:0/1035912518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717400eed0 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f7163fff640 1 -- 172.21.15.31:0/1035912518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7174003970 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c0b4f50 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f7163fff640 1 -- 172.21.15.31:0/1035912518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174003d20 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.666+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c0b42f0 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.667+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c156240 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.667+0000 7f7163fff640 1 -- 172.21.15.31:0/1035912518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f71740050d0 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.668+0000 7f7163fff640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f716806aed0 0x7f716806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.668+0000 7f7163fff640 1 -- 172.21.15.31:0/1035912518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7174089350 con 0x7f717c0d0b20 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.668+0000 7f717b7fe640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f716806aed0 0x7f716806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.669+0000 7f717b7fe640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f716806aed0 0x7f716806d390 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7170036000 tx=0x7f7170030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.672+0000 7f7163fff640 1 -- 172.21.15.31:0/1035912518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f71740542e0 con 0x7f717c0d0b20 2024-01-19T04:57:17.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.849+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f717c15dc80 con 0x7f716806aed0 2024-01-19T04:57:17.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.857+0000 7f7163fff640 1 -- 172.21.15.31:0/1035912518 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289660 (secure 0 0 0) 0x7f7170004140 con 0x7f716806aed0 2024-01-19T04:57:17.859 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:17.861 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:57:17.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f716806aed0 msgr2=0x7f716806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f716806aed0 0x7f716806d390 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7170036000 tx=0x7f7170030000 comp rx=0 tx=0).stop 2024-01-19T04:57:17.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f717c0d0b20 msgr2=0x7f717c0d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:17.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f717c0d0b20 0x7f717c0d0fb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f717c162590 tx=0x7f717400eb90 comp rx=0 tx=0).stop 2024-01-19T04:57:17.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 shutdown_connections 2024-01-19T04:57:17.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f716806aed0 0x7f716806d390 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f717c0d0b20 0x7f717c0d0fb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f717c154780 0x7f717c0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 --2- 172.21.15.31:0/1035912518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7164003970 0x7f717c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 >> 172.21.15.31:0/1035912518 conn(0x7f717c06abe0 msgr2=0x7f717c15cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 shutdown_connections 2024-01-19T04:57:17.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:17.860+0000 7f71821bb640 1 -- 172.21.15.31:0/1035912518 wait complete. 2024-01-19T04:57:17.871 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-01-19T04:57:17.588936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222068,"kb_used_data":5864,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752652,"statfs":{"total":773094113280,"available":772866715648,"internally_reserved":0,"allocated":6004736,"data_stored":3163332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001282"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892803+0000","last_change":"2024-01-19T04:57:13.537392+0000","last_active":"2024-01-19T04:57:14.892803+0000","last_peered":"2024-01-19T04:57:14.892803+0000","last_clean":"2024-01-19T04:57:14.892803+0000","last_became_active":"2024-01-19T04:57:13.537099+0000","last_became_peered":"2024-01-19T04:57:13.537099+0000","last_unstale":"2024-01-19T04:57:14.892803+0000","last_undegraded":"2024-01-19T04:57:14.892803+0000","last_fullsized":"2024-01-19T04:57:14.892803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:57:20.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892977+0000","last_change":"2024-01-19T04:57:13.539489+0000","last_active":"2024-01-19T04:57:14.892977+0000","last_peered":"2024-01-19T04:57:14.892977+0000","last_clean":"2024-01-19T04:57:14.892977+0000","last_became_active":"2024-01-19T04:57:13.539252+0000","last_became_peered":"2024-01-19T04:57:13.539252+0000","last_unstale":"2024-01-19T04:57:14.892977+0000","last_undegraded":"2024-01-19T04:57:14.892977+0000","last_fullsized":"2024-01-19T04:57:14.892977+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:56:18.105195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893454+0000","last_change":"2024-01-19T04:57:13.532406+0000","last_active":"2024-01-19T04:57:14.893454+0000","last_peered":"2024-01-19T04:57:14.893454+0000","last_clean":"2024-01-19T04:57:14.893454+0000","last_became_active":"2024-01-19T04:57:13.532133+0000","last_became_peered":"2024-01-19T04:57:13.532133+0000","last_unstale":"2024-01-19T04:57:14.893454+0000","last_undegraded":"2024-01-19T04:57:14.893454+0000","last_fullsized":"2024-01-19T04:57:14.893454+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:46:56.751777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892744+0000","last_change":"2024-01-19T04:57:13.545946+0000","last_active":"2024-01-19T04:57:14.892744+0000","last_peered":"2024-01-19T04:57:14.892744+0000","last_clean":"2024-01-19T04:57:14.892744+0000","last_became_active":"2024-01-19T04:57:13.545720+0000","last_became_peered":"2024-01-19T04:57:13.545720+0000","last_unstale":"2024-01-19T04:57:14.892744+0000","last_undegraded":"2024-01-19T04:57:14.892744+0000","last_fullsized":"2024-01-19T04:57:14.892744+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:10:03.023524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533678+0000","last_change":"2024-01-19T04:57:13.532071+0000","last_active":"2024-01-19T04:57:14.533678+0000","last_peered":"2024-01-19T04:57:14.533678+0000","last_clean":"2024-01-19T04:57:14.533678+0000","last_became_active":"2024-01-19T04:57:13.531631+0000","last_became_peered":"2024-01-19T04:57:13.531631+0000","last_unstale":"2024-01-19T04:57:14.533678+0000","last_undegraded":"2024-01-19T04:57:14.533678+0000","last_fullsized":"2024-01-19T04:57:14.533678+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:31:12.701025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893116+0000","last_change":"2024-01-19T04:57:13.535912+0000","last_active":"2024-01-19T04:57:14.893116+0000","last_peered":"2024-01-19T04:57:14.893116+0000","last_clean":"2024-01-19T04:57:14.893116+0000","last_became_active":"2024-01-19T04:57:13.535634+0000","last_became_peered":"2024-01-19T04:57:13.535634+0000","last_unstale":"2024-01-19T04:57:14.893116+0000","last_undegraded":"2024-01-19T04:57:14.893116+0000","last_fullsized":"2024-01-19T04:57:14.893116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:22:43.326022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893301+0000","last_change":"2024-01-19T04:57:13.532548+0000","last_active":"2024-01-19T04:57:14.893301+0000","last_peered":"2024-01-19T04:57:14.893301+0000","last_clean":"2024-01-19T04:57:14.893301+0000","last_became_active":"2024-01-19T04:57:13.532276+0000","last_became_peered":"2024-01-19T04:57:13.532276+0000","last_unstale":"2024-01-19T04:57:14.893301+0000","last_undegraded":"2024-01-19T04:57:14.893301+0000","last_fullsized":"2024-01-19T04:57:14.893301+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:18:06.300422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531491+0000","last_change":"2024-01-19T04:57:13.548515+0000","last_active":"2024-01-19T04:57:14.531491+0000","last_peered":"2024-01-19T04:57:14.531491+0000","last_clean":"2024-01-19T04:57:14.531491+0000","last_became_active":"2024-01-19T04:57:13.548038+0000","last_became_peered":"2024-01-19T04:57:13.548038+0000","last_unstale":"2024-01-19T04:57:14.531491+0000","last_undegraded":"2024-01-19T04:57:14.531491+0000","last_fullsized":"2024-01-19T04:57:14.531491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:44:05.894889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531699+0000","last_change":"2024-01-19T04:57:13.547657+0000","last_active":"2024-01-19T04:57:14.531699+0000","last_peered":"2024-01-19T04:57:14.531699+0000","last_clean":"2024-01-19T04:57:14.531699+0000","last_became_active":"2024-01-19T04:57:13.547449+0000","last_became_peered":"2024-01-19T04:57:13.547449+0000","last_unstale":"2024-01-19T04:57:14.531699+0000","last_undegraded":"2024-01-19T04:57:14.531699+0000","last_fullsized":"2024-01-19T04:57:14.531699+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:55:26.296266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892785+0000","last_change":"2024-01-19T04:57:13.535319+0000","last_active":"2024-01-19T04:57:14.892785+0000","last_peered":"2024-01-19T04:57:14.892785+0000","last_clean":"2024-01-19T04:57:14.892785+0000","last_became_active":"2024-01-19T04:57:13.535047+0000","last_became_peered":"2024-01-19T04:57:13.535047+0000","last_unstale":"2024-01-19T04:57:14.892785+0000","last_undegraded":"2024-01-19T04:57:14.892785+0000","last_fullsized":"2024-01-19T04:57:14.892785+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:00:19.567025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893148+0000","last_change":"2024-01-19T04:57:13.539845+0000","last_active":"2024-01-19T04:57:14.893148+0000","last_peered":"2024-01-19T04:57:14.893148+0000","last_clean":"2024-01-19T04:57:14.893148+0000","last_became_active":"2024-01-19T04:57:13.539560+0000","last_became_peered":"2024-01-19T04:57:13.539560+0000","last_unstale":"2024-01-19T04:57:14.893148+0000","last_undegraded":"2024-01-19T04:57:14.893148+0000","last_fullsized":"2024-01-19T04:57:14.893148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:07:02.919119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531625+0000","last_change":"2024-01-19T04:57:13.544873+0000","last_active":"2024-01-19T04:57:14.531625+0000","last_peered":"2024-01-19T04:57:14.531625+0000","last_clean":"2024-01-19T04:57:14.531625+0000","last_became_active":"2024-01-19T04:57:13.544505+0000","last_became_peered":"2024-01-19T04:57:13.544505+0000","last_unstale":"2024-01-19T04:57:14.531625+0000","last_undegraded":"2024-01-19T04:57:14.531625+0000","last_fullsized":"2024-01-19T04:57:14.531625+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:52:19.238961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893443+0000","last_change":"2024-01-19T04:57:13.533579+0000","last_active":"2024-01-19T04:57:14.893443+0000","last_peered":"2024-01-19T04:57:14.893443+0000","last_clean":"2024-01-19T04:57:14.893443+0000","last_became_active":"2024-01-19T04:57:13.533274+0000","last_became_peered":"2024-01-19T04:57:13.533274+0000","last_unstale":"2024-01-19T04:57:14.893443+0000","last_undegraded":"2024-01-19T04:57:14.893443+0000","last_fullsized":"2024-01-19T04:57:14.893443+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:52:18.114911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893529+0000","last_change":"2024-01-19T04:57:13.532453+0000","last_active":"2024-01-19T04:57:14.893529+0000","last_peered":"2024-01-19T04:57:14.893529+0000","last_clean":"2024-01-19T04:57:14.893529+0000","last_became_active":"2024-01-19T04:57:13.532205+0000","last_became_peered":"2024-01-19T04:57:13.532205+0000","last_unstale":"2024-01-19T04:57:14.893529+0000","last_undegraded":"2024-01-19T04:57:14.893529+0000","last_fullsized":"2024-01-19T04:57:14.893529+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:39:22.983394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892558+0000","last_change":"2024-01-19T04:57:13.537162+0000","last_active":"2024-01-19T04:57:14.892558+0000","last_peered":"2024-01-19T04:57:14.892558+0000","last_clean":"2024-01-19T04:57:14.892558+0000","last_became_active":"2024-01-19T04:57:13.536643+0000","last_became_peered":"2024-01-19T04:57:13.536643+0000","last_unstale":"2024-01-19T04:57:14.892558+0000","last_undegraded":"2024-01-19T04:57:14.892558+0000","last_fullsized":"2024-01-19T04:57:14.892558+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:17:41.696230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892812+0000","last_change":"2024-01-19T04:57:13.533805+0000","last_active":"2024-01-19T04:57:14.892812+0000","last_peered":"2024-01-19T04:57:14.892812+0000","last_clean":"2024-01-19T04:57:14.892812+0000","last_became_active":"2024-01-19T04:57:13.533339+0000","last_became_peered":"2024-01-19T04:57:13.533339+0000","last_unstale":"2024-01-19T04:57:14.892812+0000","last_undegraded":"2024-01-19T04:57:14.892812+0000","last_fullsized":"2024-01-19T04:57:14.892812+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:59:30.036210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.894007+0000","last_change":"2024-01-19T04:57:13.540037+0000","last_active":"2024-01-19T04:57:14.894007+0000","last_peered":"2024-01-19T04:57:14.894007+0000","last_clean":"2024-01-19T04:57:14.894007+0000","last_became_active":"2024-01-19T04:57:13.532069+0000","last_became_peered":"2024-01-19T04:57:13.532069+0000","last_unstale":"2024-01-19T04:57:14.894007+0000","last_undegraded":"2024-01-19T04:57:14.894007+0000","last_fullsized":"2024-01-19T04:57:14.894007+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:58:00.385962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892934+0000","last_change":"2024-01-19T04:57:13.542763+0000","last_active":"2024-01-19T04:57:14.892934+0000","last_peered":"2024-01-19T04:57:14.892934+0000","last_clean":"2024-01-19T04:57:14.892934+0000","last_became_active":"2024-01-19T04:57:13.542321+0000","last_became_peered":"2024-01-19T04:57:13.542321+0000","last_unstale":"2024-01-19T04:57:14.892934+0000","last_undegraded":"2024-01-19T04:57:14.892934+0000","last_fullsized":"2024-01-19T04:57:14.892934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:36:51.349085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893581+0000","last_change":"2024-01-19T04:57:13.530041+0000","last_active":"2024-01-19T04:57:14.893581+0000","last_peered":"2024-01-19T04:57:14.893581+0000","last_clean":"2024-01-19T04:57:14.893581+0000","last_became_active":"2024-01-19T04:57:13.529725+0000","last_became_peered":"2024-01-19T04:57:13.529725+0000","last_unstale":"2024-01-19T04:57:14.893581+0000","last_undegraded":"2024-01-19T04:57:14.893581+0000","last_fullsized":"2024-01-19T04:57:14.893581+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:23:44.461858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.694504+0000","last_change":"2024-01-19T04:57:13.532334+0000","last_active":"2024-01-19T04:57:14.694504+0000","last_peered":"2024-01-19T04:57:14.694504+0000","last_clean":"2024-01-19T04:57:14.694504+0000","last_became_active":"2024-01-19T04:57:13.532080+0000","last_became_peered":"2024-01-19T04:57:13.532080+0000","last_unstale":"2024-01-19T04:57:14.694504+0000","last_undegraded":"2024-01-19T04:57:14.694504+0000","last_fullsized":"2024-01-19T04:57:14.694504+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:27:57.454213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.694407+0000","last_change":"2024-01-19T04:57:13.535447+0000","last_active":"2024-01-19T04:57:14.694407+0000","last_peered":"2024-01-19T04:57:14.694407+0000","last_clean":"2024-01-19T04:57:14.694407+0000","last_became_active":"2024-01-19T04:57:13.535114+0000","last_became_peered":"2024-01-19T04:57:13.535114+0000","last_unstale":"2024-01-19T04:57:14.694407+0000","last_undegraded":"2024-01-19T04:57:14.694407+0000","last_fullsized":"2024-01-19T04:57:14.694407+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:11:41.641836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531279+0000","last_change":"2024-01-19T04:57:13.548442+0000","last_active":"2024-01-19T04:57:14.531279+0000","last_peered":"2024-01-19T04:57:14.531279+0000","last_clean":"2024-01-19T04:57:14.531279+0000","last_became_active":"2024-01-19T04:57:13.548085+0000","last_became_peered":"2024-01-19T04:57:13.548085+0000","last_unstale":"2024-01-19T04:57:14.531279+0000","last_undegraded":"2024-01-19T04:57:14.531279+0000","last_fullsized":"2024-01-19T04:57:14.531279+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:11:35.567352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892480+0000","last_change":"2024-01-19T04:57:13.537724+0000","last_active":"2024-01-19T04:57:14.892480+0000","last_peered":"2024-01-19T04:57:14.892480+0000","last_clean":"2024-01-19T04:57:14.892480+0000","last_became_active":"2024-01-19T04:57:13.537427+0000","last_became_peered":"2024-01-19T04:57:13.537427+0000","last_unstale":"2024-01-19T04:57:14.892480+0000","last_undegraded":"2024-01-19T04:57:14.892480+0000","last_fullsized":"2024-01-19T04:57:14.892480+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:26:36.483366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892800+0000","last_change":"2024-01-19T04:57:13.534333+0000","last_active":"2024-01-19T04:57:14.892800+0000","last_peered":"2024-01-19T04:57:14.892800+0000","last_clean":"2024-01-19T04:57:14.892800+0000","last_became_active":"2024-01-19T04:57:13.533058+0000","last_became_peered":"2024-01-19T04:57:13.533058+0000","last_unstale":"2024-01-19T04:57:14.892800+0000","last_undegraded":"2024-01-19T04:57:14.892800+0000","last_fullsized":"2024-01-19T04:57:14.892800+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:24:59.284637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.894106+0000","last_change":"2024-01-19T04:57:13.545730+0000","last_active":"2024-01-19T04:57:14.894106+0000","last_peered":"2024-01-19T04:57:14.894106+0000","last_clean":"2024-01-19T04:57:14.894106+0000","last_became_active":"2024-01-19T04:57:13.545450+0000","last_became_peered":"2024-01-19T04:57:13.545450+0000","last_unstale":"2024-01-19T04:57:14.894106+0000","last_undegraded":"2024-01-19T04:57:14.894106+0000","last_fullsized":"2024-01-19T04:57:14.894106+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:56:18.105195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531694+0000","last_change":"2024-01-19T04:57:13.544219+0000","last_active":"2024-01-19T04:57:14.531694+0000","last_peered":"2024-01-19T04:57:14.531694+0000","last_clean":"2024-01-19T04:57:14.531694+0000","last_became_active":"2024-01-19T04:57:13.543967+0000","last_became_peered":"2024-01-19T04:57:13.543967+0000","last_unstale":"2024-01-19T04:57:14.531694+0000","last_undegraded":"2024-01-19T04:57:14.531694+0000","last_fullsized":"2024-01-19T04:57:14.531694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:27:20.049720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.894065+0000","last_change":"2024-01-19T04:57:13.546847+0000","last_active":"2024-01-19T04:57:14.894065+0000","last_peered":"2024-01-19T04:57:14.894065+0000","last_clean":"2024-01-19T04:57:14.894065+0000","last_became_active":"2024-01-19T04:57:13.546487+0000","last_became_peered":"2024-01-19T04:57:13.546487+0000","last_unstale":"2024-01-19T04:57:14.894065+0000","last_undegraded":"2024-01-19T04:57:14.894065+0000","last_fullsized":"2024-01-19T04:57:14.894065+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:18:06.300422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.694541+0000","last_change":"2024-01-19T04:57:13.535735+0000","last_active":"2024-01-19T04:57:14.694541+0000","last_peered":"2024-01-19T04:57:14.694541+0000","last_clean":"2024-01-19T04:57:14.694541+0000","last_became_active":"2024-01-19T04:57:13.535503+0000","last_became_peered":"2024-01-19T04:57:13.535503+0000","last_unstale":"2024-01-19T04:57:14.694541+0000","last_undegraded":"2024-01-19T04:57:14.694541+0000","last_fullsized":"2024-01-19T04:57:14.694541+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:27:52.549869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893572+0000","last_change":"2024-01-19T04:57:13.530983+0000","last_active":"2024-01-19T04:57:14.893572+0000","last_peered":"2024-01-19T04:57:14.893572+0000","last_clean":"2024-01-19T04:57:14.893572+0000","last_became_active":"2024-01-19T04:57:13.530703+0000","last_became_peered":"2024-01-19T04:57:13.530703+0000","last_unstale":"2024-01-19T04:57:14.893572+0000","last_undegraded":"2024-01-19T04:57:14.893572+0000","last_fullsized":"2024-01-19T04:57:14.893572+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:17:41.696230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892697+0000","last_change":"2024-01-19T04:57:13.541005+0000","last_active":"2024-01-19T04:57:14.892697+0000","last_peered":"2024-01-19T04:57:14.892697+0000","last_clean":"2024-01-19T04:57:14.892697+0000","last_became_active":"2024-01-19T04:57:13.540742+0000","last_became_peered":"2024-01-19T04:57:13.540742+0000","last_unstale":"2024-01-19T04:57:14.892697+0000","last_undegraded":"2024-01-19T04:57:14.892697+0000","last_fullsized":"2024-01-19T04:57:14.892697+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:23:44.461858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893369+0000","last_change":"2024-01-19T04:57:13.539264+0000","last_active":"2024-01-19T04:57:14.893369+0000","last_peered":"2024-01-19T04:57:14.893369+0000","last_clean":"2024-01-19T04:57:14.893369+0000","last_became_active":"2024-01-19T04:57:13.538939+0000","last_became_peered":"2024-01-19T04:57:13.538939+0000","last_unstale":"2024-01-19T04:57:14.893369+0000","last_undegraded":"2024-01-19T04:57:14.893369+0000","last_fullsized":"2024-01-19T04:57:14.893369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:24:53.851298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533010+0000","last_change":"2024-01-19T04:57:13.535579+0000","last_active":"2024-01-19T04:57:14.533010+0000","last_peered":"2024-01-19T04:57:14.533010+0000","last_clean":"2024-01-19T04:57:14.533010+0000","last_became_active":"2024-01-19T04:57:13.534614+0000","last_became_peered":"2024-01-19T04:57:13.534614+0000","last_unstale":"2024-01-19T04:57:14.533010+0000","last_undegraded":"2024-01-19T04:57:14.533010+0000","last_fullsized":"2024-01-19T04:57:14.533010+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:22:43.326022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892413+0000","last_change":"2024-01-19T04:57:13.538216+0000","last_active":"2024-01-19T04:57:14.892413+0000","last_peered":"2024-01-19T04:57:14.892413+0000","last_clean":"2024-01-19T04:57:14.892413+0000","last_became_active":"2024-01-19T04:57:13.537976+0000","last_became_peered":"2024-01-19T04:57:13.537976+0000","last_unstale":"2024-01-19T04:57:14.892413+0000","last_undegraded":"2024-01-19T04:57:14.892413+0000","last_fullsized":"2024-01-19T04:57:14.892413+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:39:22.983394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.544954+0000","last_change":"2024-01-19T04:57:12.531520+0000","last_active":"2024-01-19T04:57:13.544954+0000","last_peered":"2024-01-19T04:57:13.544954+0000","last_clean":"2024-01-19T04:57:13.544954+0000","last_became_active":"2024-01-19T04:57:12.531249+0000","last_became_peered":"2024-01-19T04:57:12.531249+0000","last_unstale":"2024-01-19T04:57:13.544954+0000","last_undegraded":"2024-01-19T04:57:13.544954+0000","last_fullsized":"2024-01-19T04:57:13.544954+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:42:35.185688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533483+0000","last_change":"2024-01-19T04:57:13.539281+0000","last_active":"2024-01-19T04:57:14.533483+0000","last_peered":"2024-01-19T04:57:14.533483+0000","last_clean":"2024-01-19T04:57:14.533483+0000","last_became_active":"2024-01-19T04:57:13.537602+0000","last_became_peered":"2024-01-19T04:57:13.537602+0000","last_unstale":"2024-01-19T04:57:14.533483+0000","last_undegraded":"2024-01-19T04:57:14.533483+0000","last_fullsized":"2024-01-19T04:57:14.533483+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:51:24.019036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.544109+0000","last_change":"2024-01-19T04:57:12.528767+0000","last_active":"2024-01-19T04:57:13.544109+0000","last_peered":"2024-01-19T04:57:13.544109+0000","last_clean":"2024-01-19T04:57:13.544109+0000","last_became_active":"2024-01-19T04:57:12.528469+0000","last_became_peered":"2024-01-19T04:57:12.528469+0000","last_unstale":"2024-01-19T04:57:13.544109+0000","last_undegraded":"2024-01-19T04:57:13.544109+0000","last_fullsized":"2024-01-19T04:57:13.544109+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:27:51.547860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.694322+0000","last_change":"2024-01-19T04:57:13.535812+0000","last_active":"2024-01-19T04:57:14.694322+0000","last_peered":"2024-01-19T04:57:14.694322+0000","last_clean":"2024-01-19T04:57:14.694322+0000","last_became_active":"2024-01-19T04:57:13.535128+0000","last_became_peered":"2024-01-19T04:57:13.535128+0000","last_unstale":"2024-01-19T04:57:14.694322+0000","last_undegraded":"2024-01-19T04:57:14.694322+0000","last_fullsized":"2024-01-19T04:57:14.694322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:26:02.961095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.537751+0000","last_change":"2024-01-19T04:57:12.530934+0000","last_active":"2024-01-19T04:57:13.537751+0000","last_peered":"2024-01-19T04:57:13.537751+0000","last_clean":"2024-01-19T04:57:13.537751+0000","last_became_active":"2024-01-19T04:57:12.530626+0000","last_became_peered":"2024-01-19T04:57:12.530626+0000","last_unstale":"2024-01-19T04:57:13.537751+0000","last_undegraded":"2024-01-19T04:57:13.537751+0000","last_fullsized":"2024-01-19T04:57:13.537751+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:10:18.394957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531443+0000","last_change":"2024-01-19T04:57:13.545726+0000","last_active":"2024-01-19T04:57:14.531443+0000","last_peered":"2024-01-19T04:57:14.531443+0000","last_clean":"2024-01-19T04:57:14.531443+0000","last_became_active":"2024-01-19T04:57:13.545422+0000","last_became_peered":"2024-01-19T04:57:13.545422+0000","last_unstale":"2024-01-19T04:57:14.531443+0000","last_undegraded":"2024-01-19T04:57:14.531443+0000","last_fullsized":"2024-01-19T04:57:14.531443+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:46:10.393630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.548592+0000","last_change":"2024-01-19T04:57:12.528675+0000","last_active":"2024-01-19T04:57:13.548592+0000","last_peered":"2024-01-19T04:57:13.548592+0000","last_clean":"2024-01-19T04:57:13.548592+0000","last_became_active":"2024-01-19T04:57:12.528417+0000","last_became_peered":"2024-01-19T04:57:12.528417+0000","last_unstale":"2024-01-19T04:57:13.548592+0000","last_undegraded":"2024-01-19T04:57:13.548592+0000","last_fullsized":"2024-01-19T04:57:13.548592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:55:04.496797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.534019+0000","last_change":"2024-01-19T04:57:13.526544+0000","last_active":"2024-01-19T04:57:14.534019+0000","last_peered":"2024-01-19T04:57:14.534019+0000","last_clean":"2024-01-19T04:57:14.534019+0000","last_became_active":"2024-01-19T04:57:13.526220+0000","last_became_peered":"2024-01-19T04:57:13.526220+0000","last_unstale":"2024-01-19T04:57:14.534019+0000","last_undegraded":"2024-01-19T04:57:14.534019+0000","last_fullsized":"2024-01-19T04:57:14.534019+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:47:04.798175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.548961+0000","last_change":"2024-01-19T04:57:12.536297+0000","last_active":"2024-01-19T04:57:13.548961+0000","last_peered":"2024-01-19T04:57:13.548961+0000","last_clean":"2024-01-19T04:57:13.548961+0000","last_became_active":"2024-01-19T04:57:12.535728+0000","last_became_peered":"2024-01-19T04:57:12.535728+0000","last_unstale":"2024-01-19T04:57:13.548961+0000","last_undegraded":"2024-01-19T04:57:13.548961+0000","last_fullsized":"2024-01-19T04:57:13.548961+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:45:08.870190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892771+0000","last_change":"2024-01-19T04:57:13.538022+0000","last_active":"2024-01-19T04:57:14.892771+0000","last_peered":"2024-01-19T04:57:14.892771+0000","last_clean":"2024-01-19T04:57:14.892771+0000","last_became_active":"2024-01-19T04:57:13.537735+0000","last_became_peered":"2024-01-19T04:57:13.537735+0000","last_unstale":"2024-01-19T04:57:14.892771+0000","last_undegraded":"2024-01-19T04:57:14.892771+0000","last_fullsized":"2024-01-19T04:57:14.892771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:22:17.913910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.531267+0000","last_change":"2024-01-19T04:57:12.529801+0000","last_active":"2024-01-19T04:57:13.531267+0000","last_peered":"2024-01-19T04:57:13.531267+0000","last_clean":"2024-01-19T04:57:13.531267+0000","last_became_active":"2024-01-19T04:57:12.529480+0000","last_became_peered":"2024-01-19T04:57:12.529480+0000","last_unstale":"2024-01-19T04:57:13.531267+0000","last_undegraded":"2024-01-19T04:57:13.531267+0000","last_fullsized":"2024-01-19T04:57:13.531267+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:11:40.639827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531654+0000","last_change":"2024-01-19T04:57:13.544530+0000","last_active":"2024-01-19T04:57:14.531654+0000","last_peered":"2024-01-19T04:57:14.531654+0000","last_clean":"2024-01-19T04:57:14.531654+0000","last_became_active":"2024-01-19T04:57:13.544253+0000","last_became_peered":"2024-01-19T04:57:13.544253+0000","last_unstale":"2024-01-19T04:57:14.531654+0000","last_undegraded":"2024-01-19T04:57:14.531654+0000","last_fullsized":"2024-01-19T04:57:14.531654+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:16:56.482877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.524458+0000","last_change":"2024-01-19T04:57:12.527792+0000","last_active":"2024-01-19T04:57:13.524458+0000","last_peered":"2024-01-19T04:57:13.524458+0000","last_clean":"2024-01-19T04:57:13.524458+0000","last_became_active":"2024-01-19T04:57:12.527333+0000","last_became_peered":"2024-01-19T04:57:12.527333+0000","last_unstale":"2024-01-19T04:57:13.524458+0000","last_undegraded":"2024-01-19T04:57:13.524458+0000","last_fullsized":"2024-01-19T04:57:13.524458+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:52:26.166792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892959+0000","last_change":"2024-01-19T04:57:13.545497+0000","last_active":"2024-01-19T04:57:14.892959+0000","last_peered":"2024-01-19T04:57:14.892959+0000","last_clean":"2024-01-19T04:57:14.892959+0000","last_became_active":"2024-01-19T04:57:13.545245+0000","last_became_peered":"2024-01-19T04:57:13.545245+0000","last_unstale":"2024-01-19T04:57:14.892959+0000","last_undegraded":"2024-01-19T04:57:14.892959+0000","last_fullsized":"2024-01-19T04:57:14.892959+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:58:00.385962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.519373+0000","last_change":"2024-01-19T04:57:12.533676+0000","last_active":"2024-01-19T04:57:13.519373+0000","last_peered":"2024-01-19T04:57:13.519373+0000","last_clean":"2024-01-19T04:57:13.519373+0000","last_became_active":"2024-01-19T04:57:12.533399+0000","last_became_peered":"2024-01-19T04:57:12.533399+0000","last_unstale":"2024-01-19T04:57:13.519373+0000","last_undegraded":"2024-01-19T04:57:13.519373+0000","last_fullsized":"2024-01-19T04:57:13.519373+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:22:45.522332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533020+0000","last_change":"2024-01-19T04:57:13.542137+0000","last_active":"2024-01-19T04:57:14.533020+0000","last_peered":"2024-01-19T04:57:14.533020+0000","last_clean":"2024-01-19T04:57:14.533020+0000","last_became_active":"2024-01-19T04:57:13.541881+0000","last_became_peered":"2024-01-19T04:57:13.541881+0000","last_unstale":"2024-01-19T04:57:14.533020+0000","last_undegraded":"2024-01-19T04:57:14.533020+0000","last_fullsized":"2024-01-19T04:57:14.533020+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:45:09.872199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.529576+0000","last_change":"2024-01-19T04:57:12.529035+0000","last_active":"2024-01-19T04:57:13.529576+0000","last_peered":"2024-01-19T04:57:13.529576+0000","last_clean":"2024-01-19T04:57:13.529576+0000","last_became_active":"2024-01-19T04:57:12.528756+0000","last_became_peered":"2024-01-19T04:57:12.528756+0000","last_unstale":"2024-01-19T04:57:13.529576+0000","last_undegraded":"2024-01-19T04:57:13.529576+0000","last_fullsized":"2024-01-19T04:57:13.529576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:05:06.433347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533573+0000","last_change":"2024-01-19T04:57:13.524156+0000","last_active":"2024-01-19T04:57:14.533573+0000","last_peered":"2024-01-19T04:57:14.533573+0000","last_clean":"2024-01-19T04:57:14.533573+0000","last_became_active":"2024-01-19T04:57:13.523882+0000","last_became_peered":"2024-01-19T04:57:13.523882+0000","last_unstale":"2024-01-19T04:57:14.533573+0000","last_undegraded":"2024-01-19T04:57:14.533573+0000","last_fullsized":"2024-01-19T04:57:14.533573+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:19:52.802310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.519857+0000","last_change":"2024-01-19T04:57:12.528012+0000","last_active":"2024-01-19T04:57:13.519857+0000","last_peered":"2024-01-19T04:57:13.519857+0000","last_clean":"2024-01-19T04:57:13.519857+0000","last_became_active":"2024-01-19T04:57:12.527705+0000","last_became_peered":"2024-01-19T04:57:12.527705+0000","last_unstale":"2024-01-19T04:57:13.519857+0000","last_undegraded":"2024-01-19T04:57:13.519857+0000","last_fullsized":"2024-01-19T04:57:13.519857+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:35:29.152677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893784+0000","last_change":"2024-01-19T04:57:13.531916+0000","last_active":"2024-01-19T04:57:14.893784+0000","last_peered":"2024-01-19T04:57:14.893784+0000","last_clean":"2024-01-19T04:57:14.893784+0000","last_became_active":"2024-01-19T04:57:13.531419+0000","last_became_peered":"2024-01-19T04:57:13.531419+0000","last_unstale":"2024-01-19T04:57:14.893784+0000","last_undegraded":"2024-01-19T04:57:14.893784+0000","last_fullsized":"2024-01-19T04:57:14.893784+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:22:17.913910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.543720+0000","last_change":"2024-01-19T04:57:12.536575+0000","last_active":"2024-01-19T04:57:13.543720+0000","last_peered":"2024-01-19T04:57:13.543720+0000","last_clean":"2024-01-19T04:57:13.543720+0000","last_became_active":"2024-01-19T04:57:12.536301+0000","last_became_peered":"2024-01-19T04:57:12.536301+0000","last_unstale":"2024-01-19T04:57:13.543720+0000","last_undegraded":"2024-01-19T04:57:13.543720+0000","last_fullsized":"2024-01-19T04:57:13.543720+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:54:58.656701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533845+0000","last_change":"2024-01-19T04:57:13.531869+0000","last_active":"2024-01-19T04:57:14.533845+0000","last_peered":"2024-01-19T04:57:14.533845+0000","last_clean":"2024-01-19T04:57:14.533845+0000","last_became_active":"2024-01-19T04:57:13.531247+0000","last_became_peered":"2024-01-19T04:57:13.531247+0000","last_unstale":"2024-01-19T04:57:14.533845+0000","last_undegraded":"2024-01-19T04:57:14.533845+0000","last_fullsized":"2024-01-19T04:57:14.533845+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:10:47.691110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.552267+0000","last_change":"2024-01-19T04:57:12.533985+0000","last_active":"2024-01-19T04:57:13.552267+0000","last_peered":"2024-01-19T04:57:13.552267+0000","last_clean":"2024-01-19T04:57:13.552267+0000","last_became_active":"2024-01-19T04:57:12.533679+0000","last_became_peered":"2024-01-19T04:57:12.533679+0000","last_unstale":"2024-01-19T04:57:13.552267+0000","last_undegraded":"2024-01-19T04:57:13.552267+0000","last_fullsized":"2024-01-19T04:57:13.552267+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:18:05.298413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533066+0000","last_change":"2024-01-19T04:57:13.538584+0000","last_active":"2024-01-19T04:57:14.533066+0000","last_peered":"2024-01-19T04:57:14.533066+0000","last_clean":"2024-01-19T04:57:14.533066+0000","last_became_active":"2024-01-19T04:57:13.537236+0000","last_became_peered":"2024-01-19T04:57:13.537236+0000","last_unstale":"2024-01-19T04:57:14.533066+0000","last_undegraded":"2024-01-19T04:57:14.533066+0000","last_fullsized":"2024-01-19T04:57:14.533066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:05:07.435356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.549082+0000","last_change":"2024-01-19T04:57:12.539235+0000","last_active":"2024-01-19T04:57:13.549082+0000","last_peered":"2024-01-19T04:57:13.549082+0000","last_clean":"2024-01-19T04:57:13.549082+0000","last_became_active":"2024-01-19T04:57:12.538922+0000","last_became_peered":"2024-01-19T04:57:12.538922+0000","last_unstale":"2024-01-19T04:57:13.549082+0000","last_undegraded":"2024-01-19T04:57:13.549082+0000","last_fullsized":"2024-01-19T04:57:13.549082+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:31:11.699016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893710+0000","last_change":"2024-01-19T04:57:13.539622+0000","last_active":"2024-01-19T04:57:14.893710+0000","last_peered":"2024-01-19T04:57:14.893710+0000","last_clean":"2024-01-19T04:57:14.893710+0000","last_became_active":"2024-01-19T04:57:13.539320+0000","last_became_peered":"2024-01-19T04:57:13.539320+0000","last_unstale":"2024-01-19T04:57:14.893710+0000","last_undegraded":"2024-01-19T04:57:14.893710+0000","last_fullsized":"2024-01-19T04:57:14.893710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:10:03.023524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.525353+0000","last_change":"2024-01-19T04:57:12.538663+0000","last_active":"2024-01-19T04:57:13.525353+0000","last_peered":"2024-01-19T04:57:13.525353+0000","last_clean":"2024-01-19T04:57:13.525353+0000","last_became_active":"2024-01-19T04:57:12.538477+0000","last_became_peered":"2024-01-19T04:57:12.538477+0000","last_unstale":"2024-01-19T04:57:13.525353+0000","last_undegraded":"2024-01-19T04:57:13.525353+0000","last_fullsized":"2024-01-19T04:57:13.525353+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T05:51:51.599396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531527+0000","last_change":"2024-01-19T04:57:13.534585+0000","last_active":"2024-01-19T04:57:14.531527+0000","last_peered":"2024-01-19T04:57:14.531527+0000","last_clean":"2024-01-19T04:57:14.531527+0000","last_became_active":"2024-01-19T04:57:13.534313+0000","last_became_peered":"2024-01-19T04:57:13.534313+0000","last_unstale":"2024-01-19T04:57:14.531527+0000","last_undegraded":"2024-01-19T04:57:14.531527+0000","last_fullsized":"2024-01-19T04:57:14.531527+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:12:03.500252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.528970+0000","last_change":"2024-01-19T04:57:12.530530+0000","last_active":"2024-01-19T04:57:13.528970+0000","last_peered":"2024-01-19T04:57:13.528970+0000","last_clean":"2024-01-19T04:57:13.528970+0000","last_became_active":"2024-01-19T04:57:12.530314+0000","last_became_peered":"2024-01-19T04:57:12.530314+0000","last_unstale":"2024-01-19T04:57:13.528970+0000","last_undegraded":"2024-01-19T04:57:13.528970+0000","last_fullsized":"2024-01-19T04:57:13.528970+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:59:29.034201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893934+0000","last_change":"2024-01-19T04:57:13.532798+0000","last_active":"2024-01-19T04:57:14.893934+0000","last_peered":"2024-01-19T04:57:14.893934+0000","last_clean":"2024-01-19T04:57:14.893934+0000","last_became_active":"2024-01-19T04:57:13.531642+0000","last_became_peered":"2024-01-19T04:57:13.531642+0000","last_unstale":"2024-01-19T04:57:14.893934+0000","last_undegraded":"2024-01-19T04:57:14.893934+0000","last_fullsized":"2024-01-19T04:57:14.893934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:36:51.349085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.550480+0000","last_change":"2024-01-19T04:57:12.533409+0000","last_active":"2024-01-19T04:57:13.550480+0000","last_peered":"2024-01-19T04:57:13.550480+0000","last_clean":"2024-01-19T04:57:13.550480+0000","last_became_active":"2024-01-19T04:57:12.533108+0000","last_became_peered":"2024-01-19T04:57:12.533108+0000","last_unstale":"2024-01-19T04:57:13.550480+0000","last_undegraded":"2024-01-19T04:57:13.550480+0000","last_fullsized":"2024-01-19T04:57:13.550480+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:24:58.282628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531244+0000","last_change":"2024-01-19T04:57:13.536794+0000","last_active":"2024-01-19T04:57:14.531244+0000","last_peered":"2024-01-19T04:57:14.531244+0000","last_clean":"2024-01-19T04:57:14.531244+0000","last_became_active":"2024-01-19T04:57:13.536537+0000","last_became_peered":"2024-01-19T04:57:13.536537+0000","last_unstale":"2024-01-19T04:57:14.531244+0000","last_undegraded":"2024-01-19T04:57:14.531244+0000","last_fullsized":"2024-01-19T04:57:14.531244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:59:48.645690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.544998+0000","last_change":"2024-01-19T04:57:12.534929+0000","last_active":"2024-01-19T04:57:13.544998+0000","last_peered":"2024-01-19T04:57:13.544998+0000","last_clean":"2024-01-19T04:57:13.544998+0000","last_became_active":"2024-01-19T04:57:12.534617+0000","last_became_peered":"2024-01-19T04:57:12.534617+0000","last_unstale":"2024-01-19T04:57:13.544998+0000","last_undegraded":"2024-01-19T04:57:13.544998+0000","last_fullsized":"2024-01-19T04:57:13.544998+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:10:18.394957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893877+0000","last_change":"2024-01-19T04:57:13.549762+0000","last_active":"2024-01-19T04:57:14.893877+0000","last_peered":"2024-01-19T04:57:14.893877+0000","last_clean":"2024-01-19T04:57:14.893877+0000","last_became_active":"2024-01-19T04:57:13.549497+0000","last_became_peered":"2024-01-19T04:57:13.549497+0000","last_unstale":"2024-01-19T04:57:14.893877+0000","last_undegraded":"2024-01-19T04:57:14.893877+0000","last_fullsized":"2024-01-19T04:57:14.893877+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:51:24.019036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.546682+0000","last_change":"2024-01-19T04:57:12.532267+0000","last_active":"2024-01-19T04:57:13.546682+0000","last_peered":"2024-01-19T04:57:13.546682+0000","last_clean":"2024-01-19T04:57:13.546682+0000","last_became_active":"2024-01-19T04:57:12.531988+0000","last_became_peered":"2024-01-19T04:57:12.531988+0000","last_unstale":"2024-01-19T04:57:13.546682+0000","last_undegraded":"2024-01-19T04:57:13.546682+0000","last_fullsized":"2024-01-19T04:57:13.546682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:31:02.780958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533808+0000","last_change":"2024-01-19T04:57:13.533145+0000","last_active":"2024-01-19T04:57:14.533808+0000","last_peered":"2024-01-19T04:57:14.533808+0000","last_clean":"2024-01-19T04:57:14.533808+0000","last_became_active":"2024-01-19T04:57:13.532862+0000","last_became_peered":"2024-01-19T04:57:13.532862+0000","last_unstale":"2024-01-19T04:57:14.533808+0000","last_undegraded":"2024-01-19T04:57:14.533808+0000","last_fullsized":"2024-01-19T04:57:14.533808+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:20:09.806346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.548235+0000","last_change":"2024-01-19T04:57:12.536398+0000","last_active":"2024-01-19T04:57:13.548235+0000","last_peered":"2024-01-19T04:57:13.548235+0000","last_clean":"2024-01-19T04:57:13.548235+0000","last_became_active":"2024-01-19T04:57:12.536062+0000","last_became_peered":"2024-01-19T04:57:12.536062+0000","last_unstale":"2024-01-19T04:57:13.548235+0000","last_undegraded":"2024-01-19T04:57:13.548235+0000","last_fullsized":"2024-01-19T04:57:13.548235+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:16:12.367177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893567+0000","last_change":"2024-01-19T04:57:13.538640+0000","last_active":"2024-01-19T04:57:14.893567+0000","last_peered":"2024-01-19T04:57:14.893567+0000","last_clean":"2024-01-19T04:57:14.893567+0000","last_became_active":"2024-01-19T04:57:13.538144+0000","last_became_peered":"2024-01-19T04:57:13.538144+0000","last_unstale":"2024-01-19T04:57:14.893567+0000","last_undegraded":"2024-01-19T04:57:14.893567+0000","last_fullsized":"2024-01-19T04:57:14.893567+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:24:59.284637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.525765+0000","last_change":"2024-01-19T04:57:12.533627+0000","last_active":"2024-01-19T04:57:13.525765+0000","last_peered":"2024-01-19T04:57:13.525765+0000","last_clean":"2024-01-19T04:57:13.525765+0000","last_became_active":"2024-01-19T04:57:12.532994+0000","last_became_peered":"2024-01-19T04:57:12.532994+0000","last_unstale":"2024-01-19T04:57:13.525765+0000","last_undegraded":"2024-01-19T04:57:13.525765+0000","last_fullsized":"2024-01-19T04:57:13.525765+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:34.682307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533426+0000","last_change":"2024-01-19T04:57:13.524737+0000","last_active":"2024-01-19T04:57:14.533426+0000","last_peered":"2024-01-19T04:57:14.533426+0000","last_clean":"2024-01-19T04:57:14.533426+0000","last_became_active":"2024-01-19T04:57:13.524482+0000","last_became_peered":"2024-01-19T04:57:13.524482+0000","last_unstale":"2024-01-19T04:57:14.533426+0000","last_undegraded":"2024-01-19T04:57:14.533426+0000","last_fullsized":"2024-01-19T04:57:14.533426+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:24:59.284637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.538483+0000","last_change":"2024-01-19T04:57:12.521252+0000","last_active":"2024-01-19T04:57:13.538483+0000","last_peered":"2024-01-19T04:57:13.538483+0000","last_clean":"2024-01-19T04:57:13.538483+0000","last_became_active":"2024-01-19T04:57:12.520985+0000","last_became_peered":"2024-01-19T04:57:12.520985+0000","last_unstale":"2024-01-19T04:57:13.538483+0000","last_undegraded":"2024-01-19T04:57:13.538483+0000","last_fullsized":"2024-01-19T04:57:13.538483+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:01:47.725411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.694526+0000","last_change":"2024-01-19T04:57:13.531648+0000","last_active":"2024-01-19T04:57:14.694526+0000","last_peered":"2024-01-19T04:57:14.694526+0000","last_clean":"2024-01-19T04:57:14.694526+0000","last_became_active":"2024-01-19T04:57:13.531390+0000","last_became_peered":"2024-01-19T04:57:13.531390+0000","last_unstale":"2024-01-19T04:57:14.694526+0000","last_undegraded":"2024-01-19T04:57:14.694526+0000","last_fullsized":"2024-01-19T04:57:14.694526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:22:46.524342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.528083+0000","last_change":"2024-01-19T04:57:12.533408+0000","last_active":"2024-01-19T04:57:13.528083+0000","last_peered":"2024-01-19T04:57:13.528083+0000","last_clean":"2024-01-19T04:57:13.528083+0000","last_became_active":"2024-01-19T04:57:12.533093+0000","last_became_peered":"2024-01-19T04:57:12.533093+0000","last_unstale":"2024-01-19T04:57:13.528083+0000","last_undegraded":"2024-01-19T04:57:13.528083+0000","last_fullsized":"2024-01-19T04:57:13.528083+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:05:06.433347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893509+0000","last_change":"2024-01-19T04:57:13.531633+0000","last_active":"2024-01-19T04:57:14.893509+0000","last_peered":"2024-01-19T04:57:14.893509+0000","last_clean":"2024-01-19T04:57:14.893509+0000","last_became_active":"2024-01-19T04:57:13.531356+0000","last_became_peered":"2024-01-19T04:57:13.531356+0000","last_unstale":"2024-01-19T04:57:14.893509+0000","last_undegraded":"2024-01-19T04:57:14.893509+0000","last_fullsized":"2024-01-19T04:57:14.893509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T15:58:10.447277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.520598+0000","last_change":"2024-01-19T04:57:12.539231+0000","last_active":"2024-01-19T04:57:13.520598+0000","last_peered":"2024-01-19T04:57:13.520598+0000","last_clean":"2024-01-19T04:57:13.520598+0000","last_became_active":"2024-01-19T04:57:12.538238+0000","last_became_peered":"2024-01-19T04:57:12.538238+0000","last_unstale":"2024-01-19T04:57:13.520598+0000","last_undegraded":"2024-01-19T04:57:13.520598+0000","last_fullsized":"2024-01-19T04:57:13.520598+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T13:01:47.725411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.892944+0000","last_change":"2024-01-19T04:57:13.544778+0000","last_active":"2024-01-19T04:57:14.892944+0000","last_peered":"2024-01-19T04:57:14.892944+0000","last_clean":"2024-01-19T04:57:14.892944+0000","last_became_active":"2024-01-19T04:57:13.544515+0000","last_became_peered":"2024-01-19T04:57:13.544515+0000","last_unstale":"2024-01-19T04:57:14.892944+0000","last_undegraded":"2024-01-19T04:57:14.892944+0000","last_fullsized":"2024-01-19T04:57:14.892944+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:54:59.658710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.525063+0000","last_change":"2024-01-19T04:57:12.529720+0000","last_active":"2024-01-19T04:57:13.525063+0000","last_peered":"2024-01-19T04:57:13.525063+0000","last_clean":"2024-01-19T04:57:13.525063+0000","last_became_active":"2024-01-19T04:57:12.528619+0000","last_became_peered":"2024-01-19T04:57:12.528619+0000","last_unstale":"2024-01-19T04:57:13.525063+0000","last_undegraded":"2024-01-19T04:57:13.525063+0000","last_fullsized":"2024-01-19T04:57:13.525063+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:13:22.415764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533138+0000","last_change":"2024-01-19T04:57:13.535345+0000","last_active":"2024-01-19T04:57:14.533138+0000","last_peered":"2024-01-19T04:57:14.533138+0000","last_clean":"2024-01-19T04:57:14.533138+0000","last_became_active":"2024-01-19T04:57:13.534100+0000","last_became_peered":"2024-01-19T04:57:13.534100+0000","last_unstale":"2024-01-19T04:57:14.533138+0000","last_undegraded":"2024-01-19T04:57:14.533138+0000","last_fullsized":"2024-01-19T04:57:14.533138+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:52:18.114911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.547460+0000","last_change":"2024-01-19T04:57:12.536738+0000","last_active":"2024-01-19T04:57:13.547460+0000","last_peered":"2024-01-19T04:57:13.547460+0000","last_clean":"2024-01-19T04:57:13.547460+0000","last_became_active":"2024-01-19T04:57:12.536459+0000","last_became_peered":"2024-01-19T04:57:12.536459+0000","last_unstale":"2024-01-19T04:57:13.547460+0000","last_undegraded":"2024-01-19T04:57:13.547460+0000","last_fullsized":"2024-01-19T04:57:13.547460+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:52:05.220944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893241+0000","last_change":"2024-01-19T04:57:13.543030+0000","last_active":"2024-01-19T04:57:14.893241+0000","last_peered":"2024-01-19T04:57:14.893241+0000","last_clean":"2024-01-19T04:57:14.893241+0000","last_became_active":"2024-01-19T04:57:13.542704+0000","last_became_peered":"2024-01-19T04:57:13.542704+0000","last_unstale":"2024-01-19T04:57:14.893241+0000","last_undegraded":"2024-01-19T04:57:14.893241+0000","last_fullsized":"2024-01-19T04:57:14.893241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:05:07.435356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.529301+0000","last_change":"2024-01-19T04:57:12.533221+0000","last_active":"2024-01-19T04:57:13.529301+0000","last_peered":"2024-01-19T04:57:13.529301+0000","last_clean":"2024-01-19T04:57:13.529301+0000","last_became_active":"2024-01-19T04:57:12.532916+0000","last_became_peered":"2024-01-19T04:57:12.532916+0000","last_unstale":"2024-01-19T04:57:13.529301+0000","last_undegraded":"2024-01-19T04:57:13.529301+0000","last_fullsized":"2024-01-19T04:57:13.529301+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:22:42.324013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531528+0000","last_change":"2024-01-19T04:57:13.543815+0000","last_active":"2024-01-19T04:57:14.531528+0000","last_peered":"2024-01-19T04:57:14.531528+0000","last_clean":"2024-01-19T04:57:14.531528+0000","last_became_active":"2024-01-19T04:57:13.543533+0000","last_became_peered":"2024-01-19T04:57:13.543533+0000","last_unstale":"2024-01-19T04:57:14.531528+0000","last_undegraded":"2024-01-19T04:57:14.531528+0000","last_fullsized":"2024-01-19T04:57:14.531528+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T11:44:03.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.519899+0000","last_change":"2024-01-19T04:57:12.529195+0000","last_active":"2024-01-19T04:57:13.519899+0000","last_peered":"2024-01-19T04:57:13.519899+0000","last_clean":"2024-01-19T04:57:13.519899+0000","last_became_active":"2024-01-19T04:57:12.528943+0000","last_became_peered":"2024-01-19T04:57:12.528943+0000","last_unstale":"2024-01-19T04:57:13.519899+0000","last_undegraded":"2024-01-19T04:57:13.519899+0000","last_fullsized":"2024-01-19T04:57:13.519899+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T12:27:51.547860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893120+0000","last_change":"2024-01-19T04:57:13.547533+0000","last_active":"2024-01-19T04:57:14.893120+0000","last_peered":"2024-01-19T04:57:14.893120+0000","last_clean":"2024-01-19T04:57:14.893120+0000","last_became_active":"2024-01-19T04:57:13.547263+0000","last_became_peered":"2024-01-19T04:57:13.547263+0000","last_unstale":"2024-01-19T04:57:14.893120+0000","last_undegraded":"2024-01-19T04:57:14.893120+0000","last_fullsized":"2024-01-19T04:57:14.893120+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:51:24.019036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.530317+0000","last_change":"2024-01-19T04:57:12.535879+0000","last_active":"2024-01-19T04:57:13.530317+0000","last_peered":"2024-01-19T04:57:13.530317+0000","last_clean":"2024-01-19T04:57:13.530317+0000","last_became_active":"2024-01-19T04:57:12.535629+0000","last_became_peered":"2024-01-19T04:57:12.535629+0000","last_unstale":"2024-01-19T04:57:13.530317+0000","last_undegraded":"2024-01-19T04:57:13.530317+0000","last_fullsized":"2024-01-19T04:57:13.530317+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:51:23.017027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.533510+0000","last_change":"2024-01-19T04:57:13.537628+0000","last_active":"2024-01-19T04:57:14.533510+0000","last_peered":"2024-01-19T04:57:14.533510+0000","last_clean":"2024-01-19T04:57:14.533510+0000","last_became_active":"2024-01-19T04:57:13.537116+0000","last_became_peered":"2024-01-19T04:57:13.537116+0000","last_unstale":"2024-01-19T04:57:14.533510+0000","last_undegraded":"2024-01-19T04:57:14.533510+0000","last_fullsized":"2024-01-19T04:57:14.533510+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:18:06.300422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.528220+0000","last_change":"2024-01-19T04:57:12.528224+0000","last_active":"2024-01-19T04:57:13.528220+0000","last_peered":"2024-01-19T04:57:13.528220+0000","last_clean":"2024-01-19T04:57:13.528220+0000","last_became_active":"2024-01-19T04:57:12.527926+0000","last_became_peered":"2024-01-19T04:57:12.527926+0000","last_unstale":"2024-01-19T04:57:13.528220+0000","last_undegraded":"2024-01-19T04:57:13.528220+0000","last_fullsized":"2024-01-19T04:57:13.528220+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T08:52:17.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.893316+0000","last_change":"2024-01-19T04:57:13.536726+0000","last_active":"2024-01-19T04:57:14.893316+0000","last_peered":"2024-01-19T04:57:14.893316+0000","last_clean":"2024-01-19T04:57:14.893316+0000","last_became_active":"2024-01-19T04:57:13.536444+0000","last_became_peered":"2024-01-19T04:57:13.536444+0000","last_unstale":"2024-01-19T04:57:14.893316+0000","last_undegraded":"2024-01-19T04:57:14.893316+0000","last_fullsized":"2024-01-19T04:57:14.893316+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:19:52.802310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.519570+0000","last_change":"2024-01-19T04:57:12.530962+0000","last_active":"2024-01-19T04:57:13.519570+0000","last_peered":"2024-01-19T04:57:13.519570+0000","last_clean":"2024-01-19T04:57:13.519570+0000","last_became_active":"2024-01-19T04:57:12.530658+0000","last_became_peered":"2024-01-19T04:57:12.530658+0000","last_unstale":"2024-01-19T04:57:13.519570+0000","last_undegraded":"2024-01-19T04:57:13.519570+0000","last_fullsized":"2024-01-19T04:57:13.519570+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T06:26:01.959086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.531785+0000","last_change":"2024-01-19T04:57:13.544386+0000","last_active":"2024-01-19T04:57:14.531785+0000","last_peered":"2024-01-19T04:57:14.531785+0000","last_clean":"2024-01-19T04:57:14.531785+0000","last_became_active":"2024-01-19T04:57:13.543822+0000","last_became_peered":"2024-01-19T04:57:13.543822+0000","last_unstale":"2024-01-19T04:57:14.531785+0000","last_undegraded":"2024-01-19T04:57:14.531785+0000","last_fullsized":"2024-01-19T04:57:14.531785+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T10:17:04.124583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.519920+0000","last_change":"2024-01-19T04:57:12.524386+0000","last_active":"2024-01-19T04:57:13.519920+0000","last_peered":"2024-01-19T04:57:13.519920+0000","last_clean":"2024-01-19T04:57:13.519920+0000","last_became_active":"2024-01-19T04:57:12.524084+0000","last_became_peered":"2024-01-19T04:57:12.524084+0000","last_unstale":"2024-01-19T04:57:13.519920+0000","last_undegraded":"2024-01-19T04:57:13.519920+0000","last_fullsized":"2024-01-19T04:57:13.519920+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T04:57:39.440631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-01-19T04:57:14.694375+0000","last_change":"2024-01-19T04:57:13.532200+0000","last_active":"2024-01-19T04:57:14.694375+0000","last_peered":"2024-01-19T04:57:14.694375+0000","last_clean":"2024-01-19T04:57:14.694375+0000","last_became_active":"2024-01-19T04:57:13.531936+0000","last_became_peered":"2024-01-19T04:57:13.531936+0000","last_unstale":"2024-01-19T04:57:14.694375+0000","last_undegraded":"2024-01-19T04:57:14.694375+0000","last_fullsized":"2024-01-19T04:57:14.694375+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:12.509740+0000","last_clean_scrub_stamp":"2024-01-19T04:57:12.509740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T14:35:30.154687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.521067+0000","last_change":"2024-01-19T04:57:12.539430+0000","last_active":"2024-01-19T04:57:13.521067+0000","last_peered":"2024-01-19T04:57:13.521067+0000","last_clean":"2024-01-19T04:57:13.521067+0000","last_became_active":"2024-01-19T04:57:12.538916+0000","last_became_peered":"2024-01-19T04:57:12.538916+0000","last_unstale":"2024-01-19T04:57:13.521067+0000","last_undegraded":"2024-01-19T04:57:13.521067+0000","last_fullsized":"2024-01-19T04:57:13.521067+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:57:11.507731+0000","last_clean_scrub_stamp":"2024-01-19T04:57:11.507731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T16:20:08.804336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-01-19T04:57:13.528432+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:57:13.528432+0000","last_peered":"2024-01-19T04:57:13.528432+0000","last_clean":"2024-01-19T04:57:13.528432+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:57:13.528432+0000","last_undegraded":"2024-01-19T04:57:13.528432+0000","last_fullsized":"2024-01-19T04:57:13.528432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":837714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73099999999999998,"5min":0.753,"15min":0.753},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68200000000000005,"5min":0.745,"15min":0.745},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68500000000000005,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.621},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.504},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":602112,"data_stored":247874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.312,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.629},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.69999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.59899999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68799999999999994,"5min":0.876,"15min":0.876},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.69499999999999995,"5min":0.754,"15min":0.754},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.877,"15min":0.877},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":247222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92800000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.504,"15min":0.505},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58199999999999996}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47499999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.52800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":247804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98299999999999998,"5min":1.103,"15min":1.103},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84799999999999998,"5min":1.2,"15min":1.2},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0049999999999999,"5min":1.321,"15min":1.321},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":1.046,"15min":1.046},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.48499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51200000000000001}]},{"osd":3,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0309999999999999,"5min":1.258,"15min":1.258},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.45400000000000001}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.497},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":610304,"data_stored":248856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.318,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.34499999999999997}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53200000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.317},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.88,"15min":0.88},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.755,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80300000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.626}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71299999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81999999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":606208,"data_stored":248834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67700000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":1.47,"15min":1.47},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":1.006,"15min":1.006},"last":0.379}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90900000000000003,"5min":1.355,"15min":1.355},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.628},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82999999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.747},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81899999999999995,"5min":0.995,"15min":0.995},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":602112,"data_stored":247382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56299999999999994,"5min":1.397,"15min":1.397},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.23799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.312},{"interface":"front","average":{"1min":0.373,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":1.423,"15min":1.423},"last":0.35399999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66700000000000004,"5min":1.446,"15min":1.446},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.432},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.374}]},{"osd":4,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":1.619,"15min":1.619},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":1.585,"15min":1.585},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":1.333,"15min":1.333},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.621}]},{"osd":7,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83899999999999997,"5min":1.367,"15min":1.367},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.624},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":837646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.36899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46400000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.439}]},{"osd":2,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.42299999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.621}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81200000000000006,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.044,"15min":1.044},"last":0.68100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:57:17.875 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-19T04:57:17.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-19T04:57:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.020+0000 7f3c3fdf8640 1 Processor -- start 2024-01-19T04:57:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.020+0000 7f3c3fdf8640 1 -- start start 2024-01-19T04:57:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3fdf8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d3f10 0x7f3c380d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3fdf8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c380d1e40 0x7f3c380cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3fdf8640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c380d1450 con 0x7f3c3815d5c0 2024-01-19T04:57:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3fdf8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c380d22d0 con 0x7f3c380d1e40 2024-01-19T04:57:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3fdf8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380cf240 con 0x7f3c380d3f10 2024-01-19T04:57:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3e395640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c380d1e40 0x7f3c380cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3e395640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c380d1e40 0x7f3c380cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47848/0 (socket says 172.21.15.31:47848) 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.021+0000 7f3c3e395640 1 -- 172.21.15.31:0/2213723794 learned_addr learned my addr 172.21.15.31:0/2213723794 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.022+0000 7f3c3d393640 1 --2- 172.21.15.31:0/2213723794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d3f10 0x7f3c380d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.022+0000 7f3c3e395640 1 -- 172.21.15.31:0/2213723794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d3f10 msgr2=0x7f3c380d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.022+0000 7f3c3e395640 1 --2- 172.21.15.31:0/2213723794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d3f10 0x7f3c380d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.022+0000 7f3c3e395640 1 -- 172.21.15.31:0/2213723794 >> v1:172.21.15.137:6789/0 conn(0x7f3c3815d5c0 legacy=0x7f3c380cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.022+0000 7f3c3e395640 1 -- 172.21.15.31:0/2213723794 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c380ce6c0 con 0x7f3c380d1e40 2024-01-19T04:57:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.022+0000 7f3c3e395640 1 --2- 172.21.15.31:0/2213723794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c380d1e40 0x7f3c380cfae0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009870 tx=0x7f3c3002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd92ec83fb83c065 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3cb92640 1 -- 172.21.15.31:0/2213723794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30030840 con 0x7f3c380d1e40 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3cb92640 1 -- 172.21.15.31:0/2213723794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c300309a0 con 0x7f3c380d1e40 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3cb92640 1 -- 172.21.15.31:0/2213723794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30030d30 con 0x7f3c380d1e40 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2213723794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c380d1e40 msgr2=0x7f3c380cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2213723794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c380d1e40 0x7f3c380cfae0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009870 tx=0x7f3c3002b980 comp rx=0 tx=0).stop 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2213723794 shutdown_connections 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2213723794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c380d1e40 0x7f3c380cfae0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2213723794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d3f10 0x7f3c380d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2213723794 >> 172.21.15.31:0/2213723794 conn(0x7f3c3809c600 msgr2=0x7f3c380d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2213723794 shutdown_connections 2024-01-19T04:57:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.023+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2213723794 wait complete. 2024-01-19T04:57:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.024+0000 7f3c3fdf8640 1 Processor -- start 2024-01-19T04:57:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.024+0000 7f3c3fdf8640 1 -- start start 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3fdf8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d1e40 0x7f3c380cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3fdf8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c380d3f10 0x7f3c381592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3fdf8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3815d5c0 0x7f3c38156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3fdf8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38157280 con 0x7f3c3815d5c0 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3fdf8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38158250 con 0x7f3c380d3f10 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3fdf8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38158530 con 0x7f3c380d1e40 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3e395640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3815d5c0 0x7f3c38156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3e395640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3815d5c0 0x7f3c38156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47852/0 (socket says 172.21.15.31:47852) 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3e395640 1 -- 172.21.15.31:0/2849590789 learned_addr learned my addr 172.21.15.31:0/2849590789 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3db94640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d1e40 0x7f3c380cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3d393640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c380d3f10 0x7f3c381592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3e395640 1 -- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d1e40 msgr2=0x7f3c380cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.025+0000 7f3c3e395640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d1e40 0x7f3c380cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c3e395640 1 -- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c380d3f10 msgr2=0x7f3c381592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c3e395640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c380d3f10 0x7f3c381592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c3e395640 1 -- 172.21.15.31:0/2849590789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300094d0 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c3d393640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c380d3f10 0x7f3c381592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c3e395640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3815d5c0 0x7f3c38156d40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009870 tx=0x7f3c30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c367fc640 1 -- 172.21.15.31:0/2849590789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30039a90 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c367fc640 1 -- 172.21.15.31:0/2849590789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c30039c10 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c38158810 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c367fc640 1 -- 172.21.15.31:0/2849590789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30043410 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.026+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c38158dd0 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.027+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c38074900 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.027+0000 7f3c367fc640 1 -- 172.21.15.31:0/2849590789 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3c3004c3b0 con 0x7f3c3815d5c0 2024-01-19T04:57:18.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.028+0000 7f3c367fc640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3c2406aed0 0x7f3c2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.028+0000 7f3c367fc640 1 -- 172.21.15.31:0/2849590789 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c300ab8e0 con 0x7f3c3815d5c0 2024-01-19T04:57:18.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.028+0000 7f3c3db94640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3c2406aed0 0x7f3c2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.028+0000 7f3c3db94640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3c2406aed0 0x7f3c2406d390 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3c28001f10 tx=0x7f3c28009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:18.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.032+0000 7f3c367fc640 1 -- 172.21.15.31:0/2849590789 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3c300768f0 con 0x7f3c3815d5c0 2024-01-19T04:57:18.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.280+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3c380d3cf0 con 0x7f3c3815d5c0 2024-01-19T04:57:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.282+0000 7f3c367fc640 1 -- 172.21.15.31:0/2849590789 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3c300340b0 con 0x7f3c3815d5c0 2024-01-19T04:57:18.282 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:57:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:57:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2024-01-19T04:57:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:57:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-19T04:57:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3c2406aed0 msgr2=0x7f3c2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3c2406aed0 0x7f3c2406d390 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3c28001f10 tx=0x7f3c28009040 comp rx=0 tx=0).stop 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3815d5c0 msgr2=0x7f3c38156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3815d5c0 0x7f3c38156d40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009870 tx=0x7f3c30004730 comp rx=0 tx=0).stop 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 shutdown_connections 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3c2406aed0 0x7f3c2406d390 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3815d5c0 0x7f3c38156d40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c380d3f10 0x7f3c381592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 --2- 172.21.15.31:0/2849590789 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c380d1e40 0x7f3c380cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 >> 172.21.15.31:0/2849590789 conn(0x7f3c3809c600 msgr2=0x7f3c380b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 shutdown_connections 2024-01-19T04:57:18.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.284+0000 7f3c3fdf8640 1 -- 172.21.15.31:0/2849590789 wait complete. 2024-01-19T04:57:18.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:57:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.395+0000 7fe166ba9640 1 Processor -- start 2024-01-19T04:57:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.395+0000 7fe166ba9640 1 -- start start 2024-01-19T04:57:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe166ba9640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe16006e8d0 0x7fe1600d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe166ba9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe166ba9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe16006ecb0 0x7fe1600cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe166ba9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1600d0040 con 0x7fe16006ecb0 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe166ba9640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16015c800 con 0x7fe16006e8d0 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe166ba9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1600cd500 con 0x7fe1600cee00 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe165146640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe16006ecb0 0x7fe1600cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe165146640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe16006ecb0 0x7fe1600cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47868/0 (socket says 172.21.15.31:47868) 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.396+0000 7fe165146640 1 -- 172.21.15.31:0/3654343840 learned_addr learned my addr 172.21.15.31:0/3654343840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.397+0000 7fe164945640 1 --2- 172.21.15.31:0/3654343840 >> v2:172.21.15.137:3300/0 conn(0x7fe16006e8d0 0x7fe1600d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.397+0000 7fe15ffff640 1 --2- 172.21.15.31:0/3654343840 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.397+0000 7fe165146640 1 -- 172.21.15.31:0/3654343840 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 msgr2=0x7fe16015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.397+0000 7fe165146640 1 --2- 172.21.15.31:0/3654343840 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.397+0000 7fe165146640 1 -- 172.21.15.31:0/3654343840 >> v2:172.21.15.137:3300/0 conn(0x7fe16006e8d0 msgr2=0x7fe1600d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.397+0000 7fe165146640 1 --2- 172.21.15.31:0/3654343840 >> v2:172.21.15.137:3300/0 conn(0x7fe16006e8d0 0x7fe1600d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.397+0000 7fe165146640 1 -- 172.21.15.31:0/3654343840 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1600cb800 con 0x7fe16006ecb0 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe165146640 1 --2- 172.21.15.31:0/3654343840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600cd9e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe1600ced90 tx=0x7fe15802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4713af211e4fd20b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe15f7fe640 1 -- 172.21.15.31:0/3654343840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580317e0 con 0x7fe16006ecb0 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe15f7fe640 1 -- 172.21.15.31:0/3654343840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158031940 con 0x7fe16006ecb0 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe15f7fe640 1 -- 172.21.15.31:0/3654343840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158031cd0 con 0x7fe16006ecb0 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 -- 172.21.15.31:0/3654343840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 msgr2=0x7fe1600cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 --2- 172.21.15.31:0/3654343840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600cd9e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe1600ced90 tx=0x7fe15802b880 comp rx=0 tx=0).stop 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 -- 172.21.15.31:0/3654343840 shutdown_connections 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 --2- 172.21.15.31:0/3654343840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600cd9e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 --2- 172.21.15.31:0/3654343840 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 --2- 172.21.15.31:0/3654343840 >> v2:172.21.15.137:3300/0 conn(0x7fe16006e8d0 0x7fe1600d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 -- 172.21.15.31:0/3654343840 >> 172.21.15.31:0/3654343840 conn(0x7fe1600b2c60 msgr2=0x7fe1600b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 -- 172.21.15.31:0/3654343840 shutdown_connections 2024-01-19T04:57:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.398+0000 7fe166ba9640 1 -- 172.21.15.31:0/3654343840 wait complete. 2024-01-19T04:57:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.399+0000 7fe166ba9640 1 Processor -- start 2024-01-19T04:57:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.399+0000 7fe166ba9640 1 -- start start 2024-01-19T04:57:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe166ba9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe16006e8d0 0x7fe16014b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe166ba9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe166ba9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe166ba9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1600b4c10 con 0x7fe16006ecb0 2024-01-19T04:57:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe166ba9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1600b4650 con 0x7fe16006e8d0 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe166ba9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1600b4930 con 0x7fe1600cee00 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47884/0 (socket says 172.21.15.31:47884) 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe15ffff640 1 -- 172.21.15.31:0/2841070205 learned_addr learned my addr 172.21.15.31:0/2841070205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe164945640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe16006e8d0 0x7fe16014b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.400+0000 7fe165146640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15ffff640 1 -- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 msgr2=0x7fe16015dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15ffff640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15ffff640 1 -- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe16006e8d0 msgr2=0x7fe16014b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15ffff640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe16006e8d0 0x7fe16014b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15ffff640 1 -- 172.21.15.31:0/2841070205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580094d0 con 0x7fe16006ecb0 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15ffff640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600b3d50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe150007ec0 tx=0x7fe15000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15d7fa640 1 -- 172.21.15.31:0/2841070205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150003b90 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15d7fa640 1 -- 172.21.15.31:0/2841070205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe150003cf0 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe15d7fa640 1 -- 172.21.15.31:0/2841070205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150003ea0 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1600c8c30 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.401+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1600b4290 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.402+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe160074a80 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.403+0000 7fe15d7fa640 1 -- 172.21.15.31:0/2841070205 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe150004250 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.403+0000 7fe15d7fa640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe14006af00 0x7fe14006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.403+0000 7fe15d7fa640 1 -- 172.21.15.31:0/2841070205 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe150089a10 con 0x7fe16006ecb0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.403+0000 7fe164945640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe14006af00 0x7fe14006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.404+0000 7fe164945640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe14006af00 0x7fe14006d3c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe148003610 tx=0x7fe148008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:18.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.407+0000 7fe15d7fa640 1 -- 172.21.15.31:0/2841070205 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe150054900 con 0x7fe16006ecb0 2024-01-19T04:57:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.622+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1600696e0 con 0x7fe16006ecb0 2024-01-19T04:57:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.623+0000 7fe15d7fa640 1 -- 172.21.15.31:0/2841070205 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fe15001a090 con 0x7fe16006ecb0 2024-01-19T04:57:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:18.623 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/3002045021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3002045021},{"type":"v1","addr":"172.21.15.137:6836","nonce":3002045021}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1415057458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":1415057458},{"type":"v1","addr":"172.21.15.137:6837","nonce":1415057458}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5919,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2062734261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2062734261},{"type":"v1","addr":"172.21.15.31:6833","nonce":2062734261}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:57:12.880027+0000","modified":"2024-01-19T04:57:15.888893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1600502506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1600502506},{"type":"v1","addr":"172.21.15.31:6835","nonce":1600502506}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-19T04:57:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-01-19T04:57:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.624+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe14006af00 msgr2=0x7fe14006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.624+0000 7fe166ba9640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe14006af00 0x7fe14006d3c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe148003610 tx=0x7fe148008040 comp rx=0 tx=0).stop 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 msgr2=0x7fe1600b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600b3d50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe150007ec0 tx=0x7fe15000e4b0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 shutdown_connections 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe14006af00 0x7fe14006d3c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1600cee00 0x7fe16015dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe16006ecb0 0x7fe1600b3d50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 --2- 172.21.15.31:0/2841070205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe16006e8d0 0x7fe16014b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 >> 172.21.15.31:0/2841070205 conn(0x7fe1600b2c60 msgr2=0x7fe1600b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 shutdown_connections 2024-01-19T04:57:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:18.625+0000 7fe166ba9640 1 -- 172.21.15.31:0/2841070205 wait complete. 2024-01-19T04:57:18.880 INFO:tasks.ceph.mds.a.smithi031.stderr:2024-01-19T04:57:18.880+0000 7f6d66227640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-19T04:57:19.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-19T04:57:12.880027+0000', 'modified': '2024-01-19T04:57:15.888893+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5922': {'gid': 5922, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/1600502506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 1600502506}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 1600502506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-19T04:57:19.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-19T04:57:19.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-19T04:57:19.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.146+0000 7f4dcf9d7640 1 Processor -- start 2024-01-19T04:57:19.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.146+0000 7f4dcf9d7640 1 -- start start 2024-01-19T04:57:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcf9d7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4dc80b48e0 0x7f4dc80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcf9d7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4dc80b77c0 0x7f4dc80cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcf9d7640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc80d21f0 con 0x7f4dc80cffb0 2024-01-19T04:57:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcf9d7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc80b4cc0 con 0x7f4dc80b48e0 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcf9d7640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc80b6320 con 0x7f4dc80b77c0 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcd773640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4dc80b48e0 0x7f4dc80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcd773640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4dc80b48e0 0x7f4dc80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47886/0 (socket says 172.21.15.31:47886) 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.147+0000 7f4dcd773640 1 -- 172.21.15.31:0/174229441 learned_addr learned my addr 172.21.15.31:0/174229441 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.148+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/174229441 >> v2:172.21.15.137:3300/0 conn(0x7f4dc80b77c0 0x7f4dc80cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.148+0000 7f4dcd773640 1 -- 172.21.15.31:0/174229441 >> v2:172.21.15.137:3300/0 conn(0x7f4dc80b77c0 msgr2=0x7f4dc80cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.148+0000 7f4dcd773640 1 --2- 172.21.15.31:0/174229441 >> v2:172.21.15.137:3300/0 conn(0x7f4dc80b77c0 0x7f4dc80cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.148+0000 7f4dcd773640 1 -- 172.21.15.31:0/174229441 >> v1:172.21.15.137:6789/0 conn(0x7f4dc80cffb0 legacy=0x7f4dc80d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.148+0000 7f4dcd773640 1 -- 172.21.15.31:0/174229441 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80d0c30 con 0x7f4dc80b48e0 2024-01-19T04:57:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.148+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/174229441 >> v2:172.21.15.137:3300/0 conn(0x7f4dc80b77c0 0x7f4dc80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.148+0000 7f4dcd773640 1 --2- 172.21.15.31:0/174229441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80b48e0 0x7f4dc80cfa70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80cdab0 tx=0x7f4dc002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a5868d77b4985d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dc7fff640 1 -- 172.21.15.31:0/174229441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0031750 con 0x7f4dc80b48e0 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dc7fff640 1 -- 172.21.15.31:0/174229441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc00318b0 con 0x7f4dc80b48e0 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dc7fff640 1 -- 172.21.15.31:0/174229441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0031c40 con 0x7f4dc80b48e0 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/174229441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80b48e0 msgr2=0x7f4dc80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/174229441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80b48e0 0x7f4dc80cfa70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80cdab0 tx=0x7f4dc002b7f0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/174229441 shutdown_connections 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/174229441 >> v2:172.21.15.137:3300/0 conn(0x7f4dc80b77c0 0x7f4dc80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/174229441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80b48e0 0x7f4dc80cfa70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/174229441 >> 172.21.15.31:0/174229441 conn(0x7f4dc80d3310 msgr2=0x7f4dc80d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/174229441 shutdown_connections 2024-01-19T04:57:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.149+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/174229441 wait complete. 2024-01-19T04:57:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.150+0000 7f4dcf9d7640 1 Processor -- start 2024-01-19T04:57:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcf9d7640 1 -- start start 2024-01-19T04:57:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcf9d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc80b48e0 0x7f4dc81f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcf9d7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4dc80b77c0 0x7f4dc80cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcf9d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80cffb0 0x7f4dc80cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcf9d7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8149210 con 0x7f4dc80cffb0 2024-01-19T04:57:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcf9d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc81494f0 con 0x7f4dc80b48e0 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcf9d7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc81497d0 con 0x7f4dc80b77c0 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcdf74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80cffb0 0x7f4dc80cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcdf74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80cffb0 0x7f4dc80cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47888/0 (socket says 172.21.15.31:47888) 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.151+0000 7f4dcdf74640 1 -- 172.21.15.31:0/2939977269 learned_addr learned my addr 172.21.15.31:0/2939977269 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc80b48e0 0x7f4dc81f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4dc80b77c0 0x7f4dc80cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcdf74640 1 -- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4dc80b77c0 msgr2=0x7f4dc80cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4dc80b77c0 0x7f4dc80cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcdf74640 1 -- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc80b48e0 msgr2=0x7f4dc81f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc80b48e0 0x7f4dc81f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcdf74640 1 -- 172.21.15.31:0/2939977269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00094d0 con 0x7f4dc80cffb0 2024-01-19T04:57:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80cffb0 0x7f4dc80cc9d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4db800cea0 tx=0x7f4db800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dc5ffb640 1 -- 172.21.15.31:0/2939977269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8003b90 con 0x7f4dc80cffb0 2024-01-19T04:57:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dc5ffb640 1 -- 172.21.15.31:0/2939977269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8003d10 con 0x7f4dc80cffb0 2024-01-19T04:57:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc8149ab0 con 0x7f4dc80cffb0 2024-01-19T04:57:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dc5ffb640 1 -- 172.21.15.31:0/2939977269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8004260 con 0x7f4dc80cffb0 2024-01-19T04:57:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.152+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc81f7050 con 0x7f4dc80cffb0 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.153+0000 7f4dc5ffb640 1 -- 172.21.15.31:0/2939977269 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4db80043c0 con 0x7f4dc80cffb0 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.154+0000 7f4dc5ffb640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4db406aed0 0x7f4db406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.154+0000 7f4dc5ffb640 1 -- 172.21.15.31:0/2939977269 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4db8089f00 con 0x7f4dc80cffb0 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.154+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4db406aed0 0x7f4db406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.155+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4db406aed0 0x7f4db406d390 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0035000 tx=0x7f4dc0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.155+0000 7f4dd2599740 1 Processor -- start 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.155+0000 7f4dd2599740 1 -- start start 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93047230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb93077020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93047770 0x55fb93077560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dd2599740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb93008010 con 0x55fb93074290 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb92fb9e60 con 0x55fb93047770 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb92fb7550 con 0x55fb93074670 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93047230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93047230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47896/0 (socket says 172.21.15.31:47896) 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 -- 172.21.15.31:0/2429088231 learned_addr learned my addr 172.21.15.31:0/2429088231 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb93077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93047770 0x55fb93077560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 -- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 msgr2=0x55fb93077020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb93077020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 -- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93047770 msgr2=0x55fb93077560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93047770 0x55fb93077560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcd773640 1 -- 172.21.15.31:0/2429088231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb92faab00 con 0x55fb93074290 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.156+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93047770 0x55fb93077560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93047230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4dc002be00 tx=0x7f4dc0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89d008ed4ae72f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4daeffd640 1 -- 172.21.15.31:0/2429088231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc003dc50 con 0x55fb93074290 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4daeffd640 1 -- 172.21.15.31:0/2429088231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc003ddb0 con 0x55fb93074290 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4daeffd640 1 -- 172.21.15.31:0/2429088231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0033640 con 0x55fb93074290 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 -- 172.21.15.31:0/2429088231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 msgr2=0x55fb93047230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93047230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4dc002be00 tx=0x7f4dc0030000 comp rx=0 tx=0).stop 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 -- 172.21.15.31:0/2429088231 shutdown_connections 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93047770 0x55fb93077560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb93077020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2429088231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93047230 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 -- 172.21.15.31:0/2429088231 >> 172.21.15.31:0/2429088231 conn(0x55fb930494a0 msgr2=0x55fb93092020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 -- 172.21.15.31:0/2429088231 shutdown_connections 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.157+0000 7f4dd2599740 1 -- 172.21.15.31:0/2429088231 wait complete. 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.158+0000 7f4dd2599740 1 -- start start 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.158+0000 7f4dd2599740 1 Processor -- start 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93075490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb9307a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb930759d0 0x55fb9307ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dd2599740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb92fb2e40 con 0x55fb93074290 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb92f6c250 con 0x55fb93074670 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb92f6bef0 con 0x55fb930759d0 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dd2599740 10 client.0 _ll_register_callbacks cb 0x55fb92f70720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93075490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47912/0 (socket says 172.21.15.31:47912) 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 -- 172.21.15.31:0/2783680673 learned_addr learned my addr 172.21.15.31:0/2783680673 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb9307a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb930759d0 0x55fb9307ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 -- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb930759d0 msgr2=0x55fb9307ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb930759d0 0x55fb9307ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 -- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 msgr2=0x55fb9307a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb9307a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 -- 172.21.15.31:0/2783680673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc80d0c30 con 0x55fb93074290 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93075490 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00090b0 tx=0x7f4dc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee96de6e2ba6eea server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.159+0000 7f4da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4da7fff640 1 -- 172.21.15.31:0/2783680673 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0057070 con 0x55fb93074290 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4da7fff640 1 -- 172.21.15.31:0/2783680673 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dc0003610 con 0x55fb93074290 2024-01-19T04:57:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb92fabcd0 con 0x55fb93074290 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4da7fff640 1 -- 172.21.15.31:0/2783680673 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc00037d0 con 0x55fb93074290 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 10 client.6069 fetch_fsmap learned FSMap version 22 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55fb92fb14c0 con 0x55fb93074290 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4da7fff640 1 -- 172.21.15.31:0/2783680673 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb92fabcd0 con 0x55fb93074290 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4da7fff640 1 -- 172.21.15.31:0/2783680673 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4dc0045040 con 0x55fb93074290 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 10 client.6069 fetch_fsmap finished waiting for FSMap version 22 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 10 client.6069 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 20 client.6069 populate_metadata read hostname 'smithi031' 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930ad250 0x55fb930af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 4 client.6069 mds_command: new command op to 5922 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 --> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb92fab9f0 con 0x55fb930ad250 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930ad250 0x55fb930af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930ad250 0x55fb930af640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1664d8c19412050e server_cookie=720e05e7a514e8a7 in_seq=0 out_seq=0 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.160+0000 7f4da7fff640 10 client.6069 ms_handle_connect on v2:172.21.15.31:6834/1600502506 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4da7fff640 1 -- 172.21.15.31:0/2783680673 <== mds.0 v2:172.21.15.31:6834/1600502506 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb92fab9f0 con 0x55fb930ad250 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4da7fff640 10 client.6069 handle_command_reply: tid=0 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 client.6069 shutdown 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 msgr2=0x55fb93075490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93075490 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00090b0 tx=0x7f4dc0004730 comp rx=0 tx=0).stop 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 shutdown_connections 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930ad250 0x55fb930af640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb930759d0 0x55fb9307ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb9307a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4da7fff640 0 client.6069 ms_handle_reset on v2:172.21.15.31:6834/1600502506 2024-01-19T04:57:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2783680673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb93075490 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 >> 172.21.15.31:0/2783680673 conn(0x55fb93047770 msgr2=0x55fb9304b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 shutdown_connections 2024-01-19T04:57:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 1 -- 172.21.15.31:0/2783680673 wait complete. 2024-01-19T04:57:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.161+0000 7f4dd2599740 20 client.6069 trim_cache size 0 max 16384 2024-01-19T04:57:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.177+0000 7f4dd2599740 1 Processor -- start 2024-01-19T04:57:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.177+0000 7f4dd2599740 1 -- start start 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93047770 0x55fb93092200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb930b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb93079ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dd2599740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb92f31300 con 0x55fb93074290 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb92f748d0 con 0x55fb93074670 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb930b4480 con 0x55fb93047770 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb930b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb930b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47914/0 (socket says 172.21.15.31:47914) 2024-01-19T04:57:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 -- 172.21.15.31:0/2711170915 learned_addr learned my addr 172.21.15.31:0/2711170915 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93047770 0x55fb93092200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb93079ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 -- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93047770 msgr2=0x55fb93092200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93047770 0x55fb93092200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 -- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 msgr2=0x55fb93079ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb93079ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 -- 172.21.15.31:0/2711170915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb93073f10 con 0x55fb93074290 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb93079ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb930b3f40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00e8b0 tx=0x7f4dbc00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d1cdf9f90d2da4 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dae7fc640 1 -- 172.21.15.31:0/2711170915 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc006cd0 con 0x55fb93074290 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dae7fc640 1 -- 172.21.15.31:0/2711170915 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc01d070 con 0x55fb93074290 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.178+0000 7f4dae7fc640 1 -- 172.21.15.31:0/2711170915 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc004240 con 0x55fb93074290 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 -- 172.21.15.31:0/2711170915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 msgr2=0x55fb930b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb930b3f40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00e8b0 tx=0x7f4dbc00edd0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 -- 172.21.15.31:0/2711170915 shutdown_connections 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074670 0x55fb93079ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93074290 0x55fb930b3f40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2711170915 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93047770 0x55fb93092200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 -- 172.21.15.31:0/2711170915 >> 172.21.15.31:0/2711170915 conn(0x55fb93042d80 msgr2=0x55fb930733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 -- 172.21.15.31:0/2711170915 shutdown_connections 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 -- 172.21.15.31:0/2711170915 wait complete. 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.179+0000 7f4dd2599740 1 -- start start 2024-01-19T04:57:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 1 Processor -- start 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93047770 0x55fb930fd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074290 0x55fb9304b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb9304b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb93062a10 con 0x55fb93047770 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb93074ae0 con 0x55fb93074290 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb93094420 con 0x55fb93074670 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.180+0000 7f4dd2599740 10 client.0 _ll_register_callbacks cb 0x7f4dc004c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93047770 0x55fb930fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93047770 0x55fb930fd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47920/0 (socket says 172.21.15.31:47920) 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 -- 172.21.15.31:0/2525233617 learned_addr learned my addr 172.21.15.31:0/2525233617 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcdf74640 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb9304b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074290 0x55fb9304b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 -- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 msgr2=0x55fb9304b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb9304b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 -- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074290 msgr2=0x55fb9304b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074290 0x55fb9304b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 -- 172.21.15.31:0/2525233617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc00ea40 con 0x55fb93047770 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dcd773640 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93047770 0x55fb930fd830 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0031b70 tx=0x7f4dc0031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e8110881cc9c07 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4da7fff640 1 -- 172.21.15.31:0/2525233617 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0004290 con 0x55fb93047770 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4da7fff640 1 -- 172.21.15.31:0/2525233617 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dc005c470 con 0x55fb93047770 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dc003ddb0 con 0x55fb93047770 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4da7fff640 1 -- 172.21.15.31:0/2525233617 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc005c630 con 0x55fb93047770 2024-01-19T04:57:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dd2599740 10 client.6075 fetch_fsmap learned FSMap version 22 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55fb92f4e150 con 0x55fb93047770 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4da7fff640 1 -- 172.21.15.31:0/2525233617 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc003ddb0 con 0x55fb93047770 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4da7fff640 1 -- 172.21.15.31:0/2525233617 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4dc0045030 con 0x55fb93047770 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.181+0000 7f4dd2599740 10 client.6075 fetch_fsmap finished waiting for FSMap version 22 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 10 client.6075 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 20 client.6075 populate_metadata read hostname 'smithi031' 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930759d0 0x55fb93100660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 4 client.6075 mds_command: new command op to 5922 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 --> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb92cb26c0 con 0x55fb930759d0 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930759d0 0x55fb93100660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4da7fff640 10 client.6075 ms_handle_connect on v2:172.21.15.31:6834/1600502506 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dccf72640 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930759d0 0x55fb93100660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8ca61befaac331 server_cookie=ea5069d36169941f in_seq=0 out_seq=0 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4da7fff640 1 -- 172.21.15.31:0/2525233617 <== mds.0 v2:172.21.15.31:6834/1600502506 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fb92cb26c0 con 0x55fb930759d0 2024-01-19T04:57:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4da7fff640 10 client.6075 handle_command_reply: tid=0 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 client.6075 shutdown 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93047770 msgr2=0x55fb930fd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93047770 0x55fb930fd830 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0031b70 tx=0x7f4dc0031000 comp rx=0 tx=0).stop 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 shutdown_connections 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:6834/1600502506,v1:172.21.15.31:6835/1600502506] conn(0x55fb930759d0 0x55fb93100660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55fb93074670 0x55fb9304b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb93074290 0x55fb9304b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4da7fff640 0 client.6075 ms_handle_reset on v2:172.21.15.31:6834/1600502506 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 --2- 172.21.15.31:0/2525233617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fb93047770 0x55fb930fd830 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 >> 172.21.15.31:0/2525233617 conn(0x55fb93042d80 msgr2=0x55fb9304dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 shutdown_connections 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.182+0000 7f4dd2599740 1 -- 172.21.15.31:0/2525233617 wait complete. 2024-01-19T04:57:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dd2599740 20 client.6075 trim_cache size 0 max 16384 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "16b8b5c3-25df-408f-9c77-cd32e346926d", 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5922, 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 5.6665694630000001, 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 22, 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 50, 2024-01-19T04:57:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 50, 2024-01-19T04:57:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 11.317208489 2024-01-19T04:57:19.193 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4db406aed0 msgr2=0x7f4db406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4db406aed0 0x7f4db406d390 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0035000 tx=0x7f4dc0008030 comp rx=0 tx=0).stop 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80cffb0 msgr2=0x7f4dc80cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80cffb0 0x7f4dc80cc9d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4db800cea0 tx=0x7f4db800cf90 comp rx=0 tx=0).stop 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 shutdown_connections 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4db406aed0 0x7f4db406d390 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dc80cffb0 0x7f4dc80cc9d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4dc80b77c0 0x7f4dc80cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 --2- 172.21.15.31:0/2939977269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc80b48e0 0x7f4dc81f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 >> 172.21.15.31:0/2939977269 conn(0x7f4dc80d3310 msgr2=0x7f4dc80b8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 shutdown_connections 2024-01-19T04:57:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.183+0000 7f4dcf9d7640 1 -- 172.21.15.31:0/2939977269 wait complete. 2024-01-19T04:57:19.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:57:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.296+0000 7efbfcbe3640 1 Processor -- start 2024-01-19T04:57:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.296+0000 7efbfcbe3640 1 -- start start 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbfcbe3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efbf80d2e50 0x7efbf80b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbfcbe3640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efbf80b4d00 0x7efbf80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbfcbe3640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf8160df0 con 0x7efbf8160a10 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbfcbe3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf80b56b0 con 0x7efbf80d2e50 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbfcbe3640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf80b7300 con 0x7efbf80b4d00 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbf5d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efbf80d2e50 0x7efbf80b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbf5d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efbf80d2e50 0x7efbf80b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47936/0 (socket says 172.21.15.31:47936) 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbf5d9b640 1 -- 172.21.15.31:0/765663042 learned_addr learned my addr 172.21.15.31:0/765663042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.297+0000 7efbf6d9d640 1 --2- 172.21.15.31:0/765663042 >> v2:172.21.15.137:3300/0 conn(0x7efbf80b4d00 0x7efbf80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.298+0000 7efbf5d9b640 1 -- 172.21.15.31:0/765663042 >> v2:172.21.15.137:3300/0 conn(0x7efbf80b4d00 msgr2=0x7efbf80b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.298+0000 7efbf5d9b640 1 --2- 172.21.15.31:0/765663042 >> v2:172.21.15.137:3300/0 conn(0x7efbf80b4d00 0x7efbf80b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.298+0000 7efbf5d9b640 1 -- 172.21.15.31:0/765663042 >> v1:172.21.15.137:6789/0 conn(0x7efbf8160a10 legacy=0x7efbf80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.298+0000 7efbf5d9b640 1 -- 172.21.15.31:0/765663042 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80d1db0 con 0x7efbf80d2e50 2024-01-19T04:57:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.298+0000 7efbf5d9b640 1 --2- 172.21.15.31:0/765663042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80b8ea0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efbf80b6240 tx=0x7efbec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285c5edaef3e2901 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbf559a640 1 -- 172.21.15.31:0/765663042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec030860 con 0x7efbf80d2e50 2024-01-19T04:57:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbf559a640 1 -- 172.21.15.31:0/765663042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbec0309c0 con 0x7efbf80d2e50 2024-01-19T04:57:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbf559a640 1 -- 172.21.15.31:0/765663042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec030d50 con 0x7efbf80d2e50 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 -- 172.21.15.31:0/765663042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 msgr2=0x7efbf80b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/765663042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80b8ea0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efbf80b6240 tx=0x7efbec02b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 -- 172.21.15.31:0/765663042 shutdown_connections 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/765663042 >> v2:172.21.15.137:3300/0 conn(0x7efbf80b4d00 0x7efbf80b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/765663042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80b8ea0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 -- 172.21.15.31:0/765663042 >> 172.21.15.31:0/765663042 conn(0x7efbf8072310 msgr2=0x7efbf80cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 -- 172.21.15.31:0/765663042 shutdown_connections 2024-01-19T04:57:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.299+0000 7efbfcbe3640 1 -- 172.21.15.31:0/765663042 wait complete. 2024-01-19T04:57:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.300+0000 7efbfcbe3640 1 Processor -- start 2024-01-19T04:57:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbfcbe3640 1 -- start start 2024-01-19T04:57:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbfcbe3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf80b4d00 0x7efbf8154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbfcbe3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbfcbe3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efbf8160a10 0x7efbf81554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbfcbe3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80cc290 con 0x7efbf80d2e50 2024-01-19T04:57:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbfcbe3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80cb680 con 0x7efbf80b4d00 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbfcbe3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80cb960 con 0x7efbf8160a10 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbf5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbf5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47952/0 (socket says 172.21.15.31:47952) 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.301+0000 7efbf5d9b640 1 -- 172.21.15.31:0/2216264585 learned_addr learned my addr 172.21.15.31:0/2216264585 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf6d9d640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efbf8160a10 0x7efbf81554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf659c640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf80b4d00 0x7efbf8154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf5d9b640 1 -- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efbf8160a10 msgr2=0x7efbf81554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf5d9b640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efbf8160a10 0x7efbf81554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf5d9b640 1 -- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf80b4d00 msgr2=0x7efbf8154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf5d9b640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf80b4d00 0x7efbf8154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf5d9b640 1 -- 172.21.15.31:0/2216264585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec0094d0 con 0x7efbf80d2e50 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbf5d9b640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80ccb10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efbec02b9e0 tx=0x7efbec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbe6ffd640 1 -- 172.21.15.31:0/2216264585 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec039c90 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbe6ffd640 1 -- 172.21.15.31:0/2216264585 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbec039df0 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbe6ffd640 1 -- 172.21.15.31:0/2216264585 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec043640 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf80cbc40 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.302+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf81548b0 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.303+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf80696e0 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.304+0000 7efbe6ffd640 1 -- 172.21.15.31:0/2216264585 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efbec0437a0 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.304+0000 7efbe6ffd640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efbd806af00 0x7efbd806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.304+0000 7efbe6ffd640 1 -- 172.21.15.31:0/2216264585 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efbec03e070 con 0x7efbf80d2e50 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.304+0000 7efbf659c640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efbd806af00 0x7efbd806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.305+0000 7efbf659c640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efbd806af00 0x7efbd806d3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efbf0007fd0 tx=0x7efbf0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:19.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.308+0000 7efbe6ffd640 1 -- 172.21.15.31:0/2216264585 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efbec077660 con 0x7efbf80d2e50 2024-01-19T04:57:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.524+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efbf80b7d40 con 0x7efbf80d2e50 2024-01-19T04:57:19.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.525+0000 7efbe6ffd640 1 -- 172.21.15.31:0/2216264585 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7efbec034060 con 0x7efbf80d2e50 2024-01-19T04:57:19.524 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:57:19.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efbd806af00 msgr2=0x7efbd806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efbd806af00 0x7efbd806d3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efbf0007fd0 tx=0x7efbf0009040 comp rx=0 tx=0).stop 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 msgr2=0x7efbf80ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80ccb10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efbec02b9e0 tx=0x7efbec004730 comp rx=0 tx=0).stop 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 shutdown_connections 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efbd806af00 0x7efbd806d3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efbf8160a10 0x7efbf81554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80d2e50 0x7efbf80ccb10 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 --2- 172.21.15.31:0/2216264585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf80b4d00 0x7efbf8154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 >> 172.21.15.31:0/2216264585 conn(0x7efbf8072310 msgr2=0x7efbf80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 shutdown_connections 2024-01-19T04:57:19.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:19.527+0000 7efbfcbe3640 1 -- 172.21.15.31:0/2216264585 wait complete. 2024-01-19T04:57:19.537 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:57:19.537 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:57:19.537 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:57:19.537 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-19T04:57:19.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:19.537 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:57:19.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:57:19.538 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:57:19.538 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:19.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:19.538 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42654sec preferred_lft 42654sec 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c57:fcff:feba:867a/64 scope link 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::98a9:56ff:fef7:e409/64 scope link 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::90b1:87ff:fefd:d7a/64 scope link 2024-01-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:19.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:19.556 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:57:19.621 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:57:19.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:19.622 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:57:19.678 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:57:19.678 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:57:19.678 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:57:19.678 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:57:19.678 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:57:19.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:19.679 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:57:19.679 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:57:19.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:57:19.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:57:19.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:57:19.679 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:57:19.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:19.773 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:57:19.788 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-19T04:57:19.788 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:57:19.788 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:19.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:19.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-01-19T04:57:19.855 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-19T04:57:19.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:57:19.855 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:19.911 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:19.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:57:19.911 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:19.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:57:20.032 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:57:20.087 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:57:20.087 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:57:20.088 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:20.089 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:20.090 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:20.090 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:20.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:20.090 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:57:20.142 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:57:20.142 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-19T04:57:20.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:57:20.212 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:57:20.268 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:57:20.268 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:57:20.268 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:57:20.269 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:57:20.269 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:57:20.269 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:57:20.269 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:57:20.269 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:57:20.269 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:57:20.269 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:57:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:57:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:57:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:57:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:57:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:57:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:57:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:57:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:57:20.272 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:57:20.272 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:57:20.272 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:57:20.272 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:57:20.272 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:57:20.272 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:20.273 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:20.273 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:20.273 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:20.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:20.273 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:57:20.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-01-19T04:57:20.270+0000 7f2f41873480 -1 init, newargv = 0x5614afaf0790 newargc=15 2024-01-19T04:57:20.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[56381]: starting ceph client 2024-01-19T04:57:20.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[56381]: starting fuse 2024-01-19T04:57:20.289 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:57:20.290 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:57:20.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:57:20.290 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:57:20.291 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:57:20.291 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:57:20.291 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:57:20.291 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:20.291 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:57:20.292 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-19T04:57:20.292 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:57:20.292 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-19T04:57:20.292 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:20.292 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:57:20.293 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:57:20.293 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:57:20.293 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:57:20.293 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:20.293 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:57:20.293 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:57:20.294 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:57:20.294 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:57:20.294 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:57:20.294 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:57:20.294 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:57:20.294 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:57:20.295 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:20.295 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-01-19T04:57:20.295 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:57:20.412 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.56381.asok 2024-01-19T04:57:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:57:20.421 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56381.asok 2024-01-19T04:57:20.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:20.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56381.asok status 2024-01-19T04:57:20.559 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "56381", 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:57:20.560 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6087, 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 6087 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2496831056 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2496831056 2024-01-19T04:57:20.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.6087 192.168.0.1:0/2496831056", 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2496831056", 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 22, 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 50, 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:57:20.562 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:20.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:20.569 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:57:20.569 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:20.584 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:57:20.585 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:20.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:20.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:20.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:20.653 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2024-01-19T04:57:20.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2024-01-19T04:57:20.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.862+0000 7fac51070640 1 Processor -- start 2024-01-19T04:57:20.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.862+0000 7fac51070640 1 -- start start 2024-01-19T04:57:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac51070640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c15b6b0 0x7fac4c0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac51070640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac4c0cd2e0 0x7fac4c0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac51070640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fac4c0ceee0 0x7fac4c0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac51070640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0d03b0 con 0x7fac4c0cd2e0 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac51070640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0cce50 con 0x7fac4c0ceee0 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac51070640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0cc240 con 0x7fac4c15b6b0 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac4c0cd2e0 0x7fac4c0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac4c0cd2e0 0x7fac4c0cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59716/0 (socket says 172.21.15.31:59716) 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 -- 172.21.15.31:0/4071614023 learned_addr learned my addr 172.21.15.31:0/4071614023 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4b59e640 1 --2- 172.21.15.31:0/4071614023 >> v2:172.21.15.137:3300/0 conn(0x7fac4c0ceee0 0x7fac4c0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4ad9d640 1 --2- 172.21.15.31:0/4071614023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c15b6b0 0x7fac4c0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 -- 172.21.15.31:0/4071614023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c15b6b0 msgr2=0x7fac4c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 --2- 172.21.15.31:0/4071614023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c15b6b0 0x7fac4c0d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 -- 172.21.15.31:0/4071614023 >> v2:172.21.15.137:3300/0 conn(0x7fac4c0ceee0 msgr2=0x7fac4c0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 --2- 172.21.15.31:0/4071614023 >> v2:172.21.15.137:3300/0 conn(0x7fac4c0ceee0 0x7fac4c0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4a59c640 1 -- 172.21.15.31:0/4071614023 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0cc520 con 0x7fac4c0cd2e0 2024-01-19T04:57:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.863+0000 7fac4ad9d640 1 --2- 172.21.15.31:0/4071614023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c15b6b0 0x7fac4c0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.864+0000 7fac4a59c640 1 --2- 172.21.15.31:0/4071614023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c0cd2e0 0x7fac4c0cd980 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0724a0 tx=0x7fac4002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9b5bce90e48851 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.864+0000 7fac49d9b640 1 -- 172.21.15.31:0/4071614023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40030860 con 0x7fac4c0cd2e0 2024-01-19T04:57:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.864+0000 7fac49d9b640 1 -- 172.21.15.31:0/4071614023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac400309c0 con 0x7fac4c0cd2e0 2024-01-19T04:57:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.864+0000 7fac49d9b640 1 -- 172.21.15.31:0/4071614023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40030d50 con 0x7fac4c0cd2e0 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 -- 172.21.15.31:0/4071614023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c0cd2e0 msgr2=0x7fac4c0cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 --2- 172.21.15.31:0/4071614023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c0cd2e0 0x7fac4c0cd980 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0724a0 tx=0x7fac4002b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 -- 172.21.15.31:0/4071614023 shutdown_connections 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 --2- 172.21.15.31:0/4071614023 >> v2:172.21.15.137:3300/0 conn(0x7fac4c0ceee0 0x7fac4c0cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 --2- 172.21.15.31:0/4071614023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c0cd2e0 0x7fac4c0cd980 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 --2- 172.21.15.31:0/4071614023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c15b6b0 0x7fac4c0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 -- 172.21.15.31:0/4071614023 >> 172.21.15.31:0/4071614023 conn(0x7fac4c0d1100 msgr2=0x7fac4c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 -- 172.21.15.31:0/4071614023 shutdown_connections 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.865+0000 7fac51070640 1 -- 172.21.15.31:0/4071614023 wait complete. 2024-01-19T04:57:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.866+0000 7fac51070640 1 Processor -- start 2024-01-19T04:57:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.866+0000 7fac51070640 1 -- start start 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac51070640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac4c0cd2e0 0x7fac4c0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac51070640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c0ceee0 0x7fac4c0c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac51070640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c15b6b0 0x7fac4c14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac51070640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c09cd80 con 0x7fac4c15b6b0 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac51070640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0c9060 con 0x7fac4c0cd2e0 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac51070640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c14cb00 con 0x7fac4c0ceee0 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac4b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c15b6b0 0x7fac4c14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac4b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c15b6b0 0x7fac4c14c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59724/0 (socket says 172.21.15.31:59724) 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac4b59e640 1 -- 172.21.15.31:0/3974669837 learned_addr learned my addr 172.21.15.31:0/3974669837 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.867+0000 7fac4ad9d640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac4c0cd2e0 0x7fac4c0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac4a59c640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c0ceee0 0x7fac4c0c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac4b59e640 1 -- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c0ceee0 msgr2=0x7fac4c0c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:20.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac4b59e640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c0ceee0 0x7fac4c0c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac4b59e640 1 -- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac4c0cd2e0 msgr2=0x7fac4c0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac4b59e640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac4c0cd2e0 0x7fac4c0c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac4b59e640 1 -- 172.21.15.31:0/3974669837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400094d0 con 0x7fac4c15b6b0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac4b59e640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c15b6b0 0x7fac4c14c5c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00eed0 tx=0x7fac3c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac337fe640 1 -- 172.21.15.31:0/3974669837 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c010040 con 0x7fac4c15b6b0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac337fe640 1 -- 172.21.15.31:0/3974669837 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c014400 con 0x7fac4c15b6b0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c0c7720 con 0x7fac4c15b6b0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac337fe640 1 -- 172.21.15.31:0/3974669837 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c009ea0 con 0x7fac4c15b6b0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.868+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0c7160 con 0x7fac4c15b6b0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.869+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c074900 con 0x7fac4c15b6b0 2024-01-19T04:57:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.869+0000 7fac337fe640 1 -- 172.21.15.31:0/3974669837 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fac3c0147b0 con 0x7fac4c15b6b0 2024-01-19T04:57:20.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.870+0000 7fac337fe640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fac3806aed0 0x7fac3806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:20.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.870+0000 7fac337fe640 1 -- 172.21.15.31:0/3974669837 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac3c089de0 con 0x7fac4c15b6b0 2024-01-19T04:57:20.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.870+0000 7fac4ad9d640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fac3806aed0 0x7fac3806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:20.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.871+0000 7fac4ad9d640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fac3806aed0 0x7fac3806d390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b910 tx=0x7fac44008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:20.874+0000 7fac337fe640 1 -- 172.21.15.31:0/3974669837 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fac3c054c10 con 0x7fac4c15b6b0 2024-01-19T04:57:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.119+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7fac4c0c4690 con 0x7fac4c15b6b0 2024-01-19T04:57:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.122+0000 7fac337fe640 1 -- 172.21.15.31:0/3974669837 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v14) v1 ==== 183+0+140 (secure 0 0 0) 0x7fac3c0180e0 con 0x7fac4c15b6b0 2024-01-19T04:57:21.121 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:57:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:57:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2024-01-19T04:57:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:57:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow r pool=cephfs_data" 2024-01-19T04:57:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:57:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fac3806aed0 msgr2=0x7fac3806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fac3806aed0 0x7fac3806d390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b910 tx=0x7fac44008040 comp rx=0 tx=0).stop 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c15b6b0 msgr2=0x7fac4c14c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c15b6b0 0x7fac4c14c5c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00eed0 tx=0x7fac3c00b480 comp rx=0 tx=0).stop 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 shutdown_connections 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fac3806aed0 0x7fac3806d390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c15b6b0 0x7fac4c14c5c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac4c0ceee0 0x7fac4c0c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 --2- 172.21.15.31:0/3974669837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac4c0cd2e0 0x7fac4c0c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 >> 172.21.15.31:0/3974669837 conn(0x7fac4c0d1100 msgr2=0x7fac4c06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 shutdown_connections 2024-01-19T04:57:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.124+0000 7fac51070640 1 -- 172.21.15.31:0/3974669837 wait complete. 2024-01-19T04:57:21.134 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:21.134 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:21.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.291+0000 7f19f1870640 1 Processor -- start 2024-01-19T04:57:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.291+0000 7f19f1870640 1 -- start start 2024-01-19T04:57:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19f1870640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19ec0b3450 0x7f19ec15e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19f1870640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0cb8b0 con 0x7f19ec1580f0 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19f1870640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec15f740 con 0x7f19ec15de70 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19f1870640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec06c300 con 0x7f19ec0b3450 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19eb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19ec0b3450 0x7f19ec15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19eaffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f19ec1580f0 0x7f19ec15f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48318/0 (socket says 172.21.15.31:48318) 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19eaffd640 1 -- 172.21.15.31:0/2649549442 learned_addr learned my addr 172.21.15.31:0/2649549442 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19eb7fe640 1 -- 172.21.15.31:0/2649549442 >> v1:172.21.15.137:6789/0 conn(0x7f19ec15de70 legacy=0x7f19ec0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19eb7fe640 1 -- 172.21.15.31:0/2649549442 >> v1:172.21.15.31:6789/0 conn(0x7f19ec1580f0 legacy=0x7f19ec15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.292+0000 7f19eb7fe640 1 -- 172.21.15.31:0/2649549442 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec15e960 con 0x7f19ec0b3450 2024-01-19T04:57:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19eb7fe640 1 --2- 172.21.15.31:0/2649549442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec0b3450 0x7f19ec15e320 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009d90 tx=0x7f19e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab09b01d65eebda9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19e9ffb640 1 -- 172.21.15.31:0/2649549442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e402cc60 con 0x7f19ec0b3450 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19e9ffb640 1 -- 172.21.15.31:0/2649549442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e402cdc0 con 0x7f19ec0b3450 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19e9ffb640 1 -- 172.21.15.31:0/2649549442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4034660 con 0x7f19ec0b3450 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19f1870640 1 -- 172.21.15.31:0/2649549442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec0b3450 msgr2=0x7f19ec15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19f1870640 1 --2- 172.21.15.31:0/2649549442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec0b3450 0x7f19ec15e320 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009d90 tx=0x7f19e402beb0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19f1870640 1 -- 172.21.15.31:0/2649549442 shutdown_connections 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19f1870640 1 --2- 172.21.15.31:0/2649549442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec0b3450 0x7f19ec15e320 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19f1870640 1 -- 172.21.15.31:0/2649549442 >> 172.21.15.31:0/2649549442 conn(0x7f19ec06abe0 msgr2=0x7f19ec066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:21.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19f1870640 1 -- 172.21.15.31:0/2649549442 shutdown_connections 2024-01-19T04:57:21.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.293+0000 7f19f1870640 1 -- 172.21.15.31:0/2649549442 wait complete. 2024-01-19T04:57:21.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 Processor -- start 2024-01-19T04:57:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 -- start start 2024-01-19T04:57:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19ec0b3450 0x7f19ec0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19ec1580f0 0x7f19ec0c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec15de70 0x7f19ec1e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec152520 con 0x7f19ec15de70 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0d0eb0 con 0x7f19ec1580f0 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.295+0000 7f19f1870640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1e8300 con 0x7f19ec0b3450 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec15de70 0x7f19ec1e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec15de70 0x7f19ec1e72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59738/0 (socket says 172.21.15.31:59738) 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 -- 172.21.15.31:0/1035993271 learned_addr learned my addr 172.21.15.31:0/1035993271 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eaffd640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19ec0b3450 0x7f19ec0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19ea7fc640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19ec1580f0 0x7f19ec0c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 -- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19ec0b3450 msgr2=0x7f19ec0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19ec0b3450 0x7f19ec0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 -- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19ec1580f0 msgr2=0x7f19ec0c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19ec1580f0 0x7f19ec0c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 -- 172.21.15.31:0/1035993271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4009a70 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19eb7fe640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec15de70 0x7f19ec1e72a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009d90 tx=0x7f19e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19d3fff640 1 -- 172.21.15.31:0/1035993271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4040450 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19d3fff640 1 -- 172.21.15.31:0/1035993271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e40405d0 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec1e7d40 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19d3fff640 1 -- 172.21.15.31:0/1035993271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4040980 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.296+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec152140 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.297+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec152660 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.298+0000 7f19d3fff640 1 -- 172.21.15.31:0/1035993271 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f19e4040ae0 con 0x7f19ec15de70 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.298+0000 7f19d3fff640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f19d806aed0 0x7f19d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.298+0000 7f19d3fff640 1 -- 172.21.15.31:0/1035993271 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19e40ac120 con 0x7f19ec15de70 2024-01-19T04:57:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.298+0000 7f19eaffd640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f19d806aed0 0x7f19d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.299+0000 7f19eaffd640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f19d806aed0 0x7f19d806d390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f19ec153e80 tx=0x7f19dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.302+0000 7f19d3fff640 1 -- 172.21.15.31:0/1035993271 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f19e4077140 con 0x7f19ec15de70 2024-01-19T04:57:21.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.482+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19ec0696e0 con 0x7f19ec15de70 2024-01-19T04:57:21.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.483+0000 7f19d3fff640 1 -- 172.21.15.31:0/1035993271 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f19e407a760 con 0x7f19ec15de70 2024-01-19T04:57:21.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:21.483 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:21.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f19d806aed0 msgr2=0x7f19d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f19d806aed0 0x7f19d806d390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f19ec153e80 tx=0x7f19dc009040 comp rx=0 tx=0).stop 2024-01-19T04:57:21.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec15de70 msgr2=0x7f19ec1e72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec15de70 0x7f19ec1e72a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009d90 tx=0x7f19e4004a40 comp rx=0 tx=0).stop 2024-01-19T04:57:21.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 shutdown_connections 2024-01-19T04:57:21.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f19d806aed0 0x7f19d806d390 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ec15de70 0x7f19ec1e72a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19ec1580f0 0x7f19ec0c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 --2- 172.21.15.31:0/1035993271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f19ec0b3450 0x7f19ec0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.485+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 >> 172.21.15.31:0/1035993271 conn(0x7f19ec06abe0 msgr2=0x7f19ec15c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:21.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.486+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 shutdown_connections 2024-01-19T04:57:21.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.486+0000 7f19f1870640 1 -- 172.21.15.31:0/1035993271 wait complete. 2024-01-19T04:57:21.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:57:10.830975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:57:11.850371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:21.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.603+0000 7f53080fc640 1 Processor -- start 2024-01-19T04:57:21.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.603+0000 7f53080fc640 1 -- start start 2024-01-19T04:57:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f53080fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f530015d5d0 0x7f53000cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f53080fc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f53000d0250 0x7f53000d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f53080fc640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53000d3c60 con 0x7f5300161a30 2024-01-19T04:57:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f53080fc640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53000ce490 con 0x7f530015d5d0 2024-01-19T04:57:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f53080fc640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53000ce770 con 0x7f53000d0250 2024-01-19T04:57:21.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5305e98640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f530015d5d0 0x7f53000cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5305e98640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f530015d5d0 0x7f53000cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59752/0 (socket says 172.21.15.31:59752) 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5305e98640 1 -- 172.21.15.31:0/2278907528 learned_addr learned my addr 172.21.15.31:0/2278907528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5306699640 1 --2- 172.21.15.31:0/2278907528 >> v2:172.21.15.137:3300/0 conn(0x7f53000d0250 0x7f53000d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5305e98640 1 -- 172.21.15.31:0/2278907528 >> v2:172.21.15.137:3300/0 conn(0x7f53000d0250 msgr2=0x7f53000d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5305e98640 1 --2- 172.21.15.31:0/2278907528 >> v2:172.21.15.137:3300/0 conn(0x7f53000d0250 0x7f53000d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5305e98640 1 -- 172.21.15.31:0/2278907528 >> v1:172.21.15.137:6789/0 conn(0x7f5300161a30 legacy=0x7f53000cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5305e98640 1 -- 172.21.15.31:0/2278907528 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000cea50 con 0x7f530015d5d0 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.604+0000 7f5306699640 1 --2- 172.21.15.31:0/2278907528 >> v2:172.21.15.137:3300/0 conn(0x7f53000d0250 0x7f53000d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.605+0000 7f5305e98640 1 --2- 172.21.15.31:0/2278907528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53000cde80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f530008c710 tx=0x7f52fc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c7fd622e7c201c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.605+0000 7f5304e96640 1 -- 172.21.15.31:0/2278907528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc030860 con 0x7f530015d5d0 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.605+0000 7f5304e96640 1 -- 172.21.15.31:0/2278907528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0309c0 con 0x7f530015d5d0 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.605+0000 7f5304e96640 1 -- 172.21.15.31:0/2278907528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc030d50 con 0x7f530015d5d0 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 -- 172.21.15.31:0/2278907528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 msgr2=0x7f53000cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 --2- 172.21.15.31:0/2278907528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53000cde80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f530008c710 tx=0x7f52fc02b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 -- 172.21.15.31:0/2278907528 shutdown_connections 2024-01-19T04:57:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 --2- 172.21.15.31:0/2278907528 >> v2:172.21.15.137:3300/0 conn(0x7f53000d0250 0x7f53000d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 --2- 172.21.15.31:0/2278907528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53000cde80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 -- 172.21.15.31:0/2278907528 >> 172.21.15.31:0/2278907528 conn(0x7f530009c5e0 msgr2=0x7f53000d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 -- 172.21.15.31:0/2278907528 shutdown_connections 2024-01-19T04:57:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.606+0000 7f53080fc640 1 -- 172.21.15.31:0/2278907528 wait complete. 2024-01-19T04:57:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.607+0000 7f53080fc640 1 Processor -- start 2024-01-19T04:57:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.607+0000 7f53080fc640 1 -- start start 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53000d0250 0x7f5300156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53001579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5300161a30 0x7f5300157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300158dc0 con 0x7f530015d5d0 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53001576f0 con 0x7f53000d0250 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5300158520 con 0x7f5300161a30 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53001579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53001579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59756/0 (socket says 172.21.15.31:59756) 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 -- 172.21.15.31:0/1412806842 learned_addr learned my addr 172.21.15.31:0/1412806842 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305e98640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53000d0250 0x7f5300156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5306699640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5300161a30 0x7f5300157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 -- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5300161a30 msgr2=0x7f5300157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5300161a30 0x7f5300157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 -- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53000d0250 msgr2=0x7f5300156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53000d0250 0x7f5300156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 -- 172.21.15.31:0/1412806842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc0094d0 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f5305697640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53001579b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c3c0 tx=0x7f52f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f52f67fc640 1 -- 172.21.15.31:0/1412806842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00052d0 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f52f67fc640 1 -- 172.21.15.31:0/1412806842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0005450 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f52f67fc640 1 -- 172.21.15.31:0/1412806842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0007260 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5300158800 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.608+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5300157220 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.609+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000c6d80 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.610+0000 7f52f67fc640 1 -- 172.21.15.31:0/1412806842 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f52f00073c0 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.610+0000 7f52f67fc640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52e406aed0 0x7f52e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.610+0000 7f52f67fc640 1 -- 172.21.15.31:0/1412806842 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52f0088920 con 0x7f530015d5d0 2024-01-19T04:57:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.610+0000 7f5305e98640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52e406aed0 0x7f52e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.611+0000 7f5305e98640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52e406aed0 0x7f52e406d390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f52fc036000 tx=0x7f52fc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.614+0000 7f52f67fc640 1 -- 172.21.15.31:0/1412806842 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f53000c6d80 con 0x7f530015d5d0 2024-01-19T04:57:21.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.791+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53000c6d80 con 0x7f530015d5d0 2024-01-19T04:57:21.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.791+0000 7f52f67fc640 1 -- 172.21.15.31:0/1412806842 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53000c6d80 con 0x7f530015d5d0 2024-01-19T04:57:21.791 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52e406aed0 msgr2=0x7f52e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52e406aed0 0x7f52e406d390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f52fc036000 tx=0x7f52fc008030 comp rx=0 tx=0).stop 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 msgr2=0x7f53001579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53001579b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c3c0 tx=0x7f52f000c880 comp rx=0 tx=0).stop 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 shutdown_connections 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f52e406aed0 0x7f52e406d390 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5300161a30 0x7f5300157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f530015d5d0 0x7f53001579b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 --2- 172.21.15.31:0/1412806842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53000d0250 0x7f5300156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 >> 172.21.15.31:0/1412806842 conn(0x7f530009c5e0 msgr2=0x7f530015b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:21.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 shutdown_connections 2024-01-19T04:57:21.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.793+0000 7f53080fc640 1 -- 172.21.15.31:0/1412806842 wait complete. 2024-01-19T04:57:21.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:57:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.907+0000 7f841d51d640 1 Processor -- start 2024-01-19T04:57:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.907+0000 7f841d51d640 1 -- start start 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f841d51d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f84180cf6f0 0x7f84180add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f841d51d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84180b1d60 0x7f84180b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f841d51d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84180b0a70 con 0x7f84180cfad0 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f841d51d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84180b18d0 con 0x7f84180b1d60 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f841d51d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84180b0cc0 con 0x7f84180cf6f0 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f84177fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84180b1d60 0x7f84180b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f84167fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f84180cfad0 0x7f84180cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48322/0 (socket says 172.21.15.31:48322) 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f84177fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84180b1d60 0x7f84180b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59768/0 (socket says 172.21.15.31:59768) 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f84167fc640 1 -- 172.21.15.31:0/1713010135 learned_addr learned my addr 172.21.15.31:0/1713010135 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.908+0000 7f8416ffd640 1 --2- 172.21.15.31:0/1713010135 >> v2:172.21.15.137:3300/0 conn(0x7f84180cf6f0 0x7f84180add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.909+0000 7f84177fe640 1 -- 172.21.15.31:0/1713010135 >> v2:172.21.15.137:3300/0 conn(0x7f84180cf6f0 msgr2=0x7f84180add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.909+0000 7f84177fe640 1 --2- 172.21.15.31:0/1713010135 >> v2:172.21.15.137:3300/0 conn(0x7f84180cf6f0 0x7f84180add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.909+0000 7f84177fe640 1 -- 172.21.15.31:0/1713010135 >> v1:172.21.15.31:6789/0 conn(0x7f84180cfad0 legacy=0x7f84180cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.909+0000 7f8415ffb640 1 -- 172.21.15.31:0/1713010135 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506904564 0 0) 0x7f84180b0a70 con 0x7f84180cfad0 2024-01-19T04:57:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.909+0000 7f84177fe640 1 -- 172.21.15.31:0/1713010135 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180b0fa0 con 0x7f84180b1d60 2024-01-19T04:57:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.909+0000 7f84177fe640 1 --2- 172.21.15.31:0/1713010135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180b1d60 0x7f84180b2170 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f841806b0c0 tx=0x7f841002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc21403c0594b50 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.909+0000 7f8415ffb640 1 -- 172.21.15.31:0/1713010135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410031830 con 0x7f84180b1d60 2024-01-19T04:57:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f8415ffb640 1 -- 172.21.15.31:0/1713010135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8410031990 con 0x7f84180b1d60 2024-01-19T04:57:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f8415ffb640 1 -- 172.21.15.31:0/1713010135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410031d20 con 0x7f84180b1d60 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 -- 172.21.15.31:0/1713010135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180b1d60 msgr2=0x7f84180b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 --2- 172.21.15.31:0/1713010135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180b1d60 0x7f84180b2170 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f841806b0c0 tx=0x7f841002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 -- 172.21.15.31:0/1713010135 shutdown_connections 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 --2- 172.21.15.31:0/1713010135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180b1d60 0x7f84180b2170 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 --2- 172.21.15.31:0/1713010135 >> v2:172.21.15.137:3300/0 conn(0x7f84180cf6f0 0x7f84180add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 -- 172.21.15.31:0/1713010135 >> 172.21.15.31:0/1713010135 conn(0x7f841808afe0 msgr2=0x7f84180d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 -- 172.21.15.31:0/1713010135 shutdown_connections 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.910+0000 7f841d51d640 1 -- 172.21.15.31:0/1713010135 wait complete. 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.911+0000 7f841d51d640 1 Processor -- start 2024-01-19T04:57:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.911+0000 7f841d51d640 1 -- start start 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f841d51d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b1d60 0x7f84180c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f841d51d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84180cf6f0 0x7f84181f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f841d51d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180cfad0 0x7f84181505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f841d51d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84181f5180 con 0x7f84180cfad0 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f841d51d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418150ae0 con 0x7f84180cf6f0 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f841d51d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180c87c0 con 0x7f84180b1d60 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f84177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180cfad0 0x7f84181505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f84177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180cfad0 0x7f84181505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59782/0 (socket says 172.21.15.31:59782) 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.912+0000 7f84177fe640 1 -- 172.21.15.31:0/1470372049 learned_addr learned my addr 172.21.15.31:0/1470372049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f8416ffd640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b1d60 0x7f84180c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f84167fc640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84180cf6f0 0x7f84181f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f84177fe640 1 -- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b1d60 msgr2=0x7f84180c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f84177fe640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b1d60 0x7f84180c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f84177fe640 1 -- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84180cf6f0 msgr2=0x7f84181f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f84177fe640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84180cf6f0 0x7f84181f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f84177fe640 1 -- 172.21.15.31:0/1470372049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410009400 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f84177fe640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180cfad0 0x7f84181505a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8410034040 tx=0x7f84100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f840f7fe640 1 -- 172.21.15.31:0/1470372049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100317d0 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f840f7fe640 1 -- 172.21.15.31:0/1470372049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841003a9e0 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84180c8aa0 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f840f7fe640 1 -- 172.21.15.31:0/1470372049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841003ad90 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.913+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84180c9060 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.914+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418074900 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.914+0000 7f840f7fe640 1 -- 172.21.15.31:0/1470372049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f84100443f0 con 0x7f84180cfad0 2024-01-19T04:57:21.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.915+0000 7f840f7fe640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83f806aed0 0x7f83f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:21.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.915+0000 7f840f7fe640 1 -- 172.21.15.31:0/1470372049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84100ab9c0 con 0x7f84180cfad0 2024-01-19T04:57:21.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.915+0000 7f8416ffd640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83f806aed0 0x7f83f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:21.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.916+0000 7f8416ffd640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83f806aed0 0x7f83f806d390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8408009a60 tx=0x7f8408003990 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:21.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:21.919+0000 7f840f7fe640 1 -- 172.21.15.31:0/1470372049 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f84100769a0 con 0x7f84180cfad0 2024-01-19T04:57:22.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.097+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84180cf4d0 con 0x7f84180cfad0 2024-01-19T04:57:22.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.097+0000 7f840f7fe640 1 -- 172.21.15.31:0/1470372049 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8410079fc0 con 0x7f84180cfad0 2024-01-19T04:57:22.097 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83f806aed0 msgr2=0x7f83f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83f806aed0 0x7f83f806d390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8408009a60 tx=0x7f8408003990 comp rx=0 tx=0).stop 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180cfad0 msgr2=0x7f84181505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180cfad0 0x7f84181505a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8410034040 tx=0x7f84100046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 shutdown_connections 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83f806aed0 0x7f83f806d390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84180cfad0 0x7f84181505a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84180cf6f0 0x7f84181f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 --2- 172.21.15.31:0/1470372049 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b1d60 0x7f84180c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 >> 172.21.15.31:0/1470372049 conn(0x7f841808afe0 msgr2=0x7f84180add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 shutdown_connections 2024-01-19T04:57:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.099+0000 7f841d51d640 1 -- 172.21.15.31:0/1470372049 wait complete. 2024-01-19T04:57:22.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:57:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.214+0000 7f5f35168640 1 Processor -- start 2024-01-19T04:57:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.214+0000 7f5f35168640 1 -- start start 2024-01-19T04:57:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f35168640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 0x7f5f300d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f35168640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f300cef10 con 0x7f5f3015d700 2024-01-19T04:57:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f35168640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f300ce300 con 0x7f5f300d11b0 2024-01-19T04:57:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f35168640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300ce5e0 con 0x7f5f300d1fe0 2024-01-19T04:57:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2ed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5f3015d700 0x7f5f300d18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48336/0 (socket says 172.21.15.31:48336) 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2ed9d640 1 -- 172.21.15.31:0/3470680431 learned_addr learned my addr 172.21.15.31:0/3470680431 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115860707 0 0) 0x7f5f300cef10 con 0x7f5f3015d700 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f1c003520 con 0x7f5f3015d700 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2f59e640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 0x7f5f300d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 427640493 0 0) 0x7f5f1c003520 con 0x7f5f3015d700 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 msgr2=0x7f5f300d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 0x7f5f300d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 >> v1:172.21.15.137:6789/0 conn(0x7f5f300d11b0 legacy=0x7f5f300b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300ce8c0 con 0x7f5f3015d700 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f5f28001670 con 0x7f5f3015d700 2024-01-19T04:57:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2f59e640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 0x7f5f300d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 >> v1:172.21.15.31:6789/0 conn(0x7f5f3015d700 legacy=0x7f5f300d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 0x7f5f1c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c003f90 0x7f5f1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c004d00 0x7f5f1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300ce5e0 con 0x7f5f1c004d00 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c00bb70 con 0x7f5f1c003f90 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.215+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c00bd30 con 0x7f5f300d1fe0 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2f59e640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c004d00 0x7f5f1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2ed9d640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 0x7f5f1c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2e59c640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c003f90 0x7f5f1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2f59e640 1 -- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 msgr2=0x7f5f1c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2f59e640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300d1fe0 0x7f5f1c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2f59e640 1 -- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c003f90 msgr2=0x7f5f1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2f59e640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c003f90 0x7f5f1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.216+0000 7f5f2f59e640 1 -- 172.21.15.31:0/3470680431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c003520 con 0x7f5f1c004d00 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f2f59e640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c004d00 0x7f5f1c005180 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5f200073e0 tx=0x7f5f1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c73e199e5f36021 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2002cd30 con 0x7f5f1c004d00 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2002ce90 con 0x7f5f1c004d00 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f2dd9b640 1 -- 172.21.15.31:0/3470680431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20034640 con 0x7f5f1c004d00 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f35168640 1 -- 172.21.15.31:0/3470680431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c004d00 msgr2=0x7f5f1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f35168640 1 --2- 172.21.15.31:0/3470680431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c004d00 0x7f5f1c005180 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5f200073e0 tx=0x7f5f1c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f2f59e640 1 -- 172.21.15.31:0/3470680431 reap_dead start 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f35168640 1 -- 172.21.15.31:0/3470680431 shutdown_connections 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f35168640 1 -- 172.21.15.31:0/3470680431 >> 172.21.15.31:0/3470680431 conn(0x7f5f300d32f0 msgr2=0x7f5f300d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f35168640 1 -- 172.21.15.31:0/3470680431 shutdown_connections 2024-01-19T04:57:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.217+0000 7f5f35168640 1 -- 172.21.15.31:0/3470680431 wait complete. 2024-01-19T04:57:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.219+0000 7f5f35168640 1 Processor -- start 2024-01-19T04:57:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.219+0000 7f5f35168640 1 -- start start 2024-01-19T04:57:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f30157bf0 0x7f5f300c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300cb8a0 0x7f5f300b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f300b20e0 0x7f5f300ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300b2560 con 0x7f5f30157bf0 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300cbd20 con 0x7f5f300b20e0 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300cbfa0 con 0x7f5f300cb8a0 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f30157bf0 0x7f5f300c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f30157bf0 0x7f5f300c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59792/0 (socket says 172.21.15.31:59792) 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 -- 172.21.15.31:0/1697674182 learned_addr learned my addr 172.21.15.31:0/1697674182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 -- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300cb8a0 msgr2=0x7f5f300b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300cb8a0 0x7f5f300b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 -- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f300b20e0 msgr2=0x7f5f300ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f300b20e0 0x7f5f300ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 -- 172.21.15.31:0/1697674182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300cc280 con 0x7f5f30157bf0 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f2ed9d640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f30157bf0 0x7f5f300c8350 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5f280072c0 tx=0x7f5f28007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f277fe640 1 -- 172.21.15.31:0/1697674182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800f940 con 0x7f5f30157bf0 2024-01-19T04:57:22.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f277fe640 1 -- 172.21.15.31:0/1697674182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2800fae0 con 0x7f5f30157bf0 2024-01-19T04:57:22.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f300cc560 con 0x7f5f30157bf0 2024-01-19T04:57:22.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f277fe640 1 -- 172.21.15.31:0/1697674182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800fe90 con 0x7f5f30157bf0 2024-01-19T04:57:22.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.220+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f300ccb20 con 0x7f5f30157bf0 2024-01-19T04:57:22.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.221+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30074900 con 0x7f5f30157bf0 2024-01-19T04:57:22.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.222+0000 7f5f277fe640 1 -- 172.21.15.31:0/1697674182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5f2801d460 con 0x7f5f30157bf0 2024-01-19T04:57:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.222+0000 7f5f277fe640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f1006aed0 0x7f5f1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.222+0000 7f5f277fe640 1 -- 172.21.15.31:0/1697674182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f28089930 con 0x7f5f30157bf0 2024-01-19T04:57:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.223+0000 7f5f2e59c640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f1006aed0 0x7f5f1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.223+0000 7f5f2e59c640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f1006aed0 0x7f5f1006d390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f300d0290 tx=0x7f5f18009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.227+0000 7f5f277fe640 1 -- 172.21.15.31:0/1697674182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5f28054760 con 0x7f5f30157bf0 2024-01-19T04:57:22.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.404+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f300d16b0 con 0x7f5f30157bf0 2024-01-19T04:57:22.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.404+0000 7f5f277fe640 1 -- 172.21.15.31:0/1697674182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f28057d80 con 0x7f5f30157bf0 2024-01-19T04:57:22.404 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f1006aed0 msgr2=0x7f5f1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f1006aed0 0x7f5f1006d390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f300d0290 tx=0x7f5f18009040 comp rx=0 tx=0).stop 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f30157bf0 msgr2=0x7f5f300c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f30157bf0 0x7f5f300c8350 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5f280072c0 tx=0x7f5f28007780 comp rx=0 tx=0).stop 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 shutdown_connections 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f1006aed0 0x7f5f1006d390 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f300b20e0 0x7f5f300ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f300cb8a0 0x7f5f300b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 --2- 172.21.15.31:0/1697674182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f30157bf0 0x7f5f300c8350 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 >> 172.21.15.31:0/1697674182 conn(0x7f5f300d32f0 msgr2=0x7f5f300d38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:22.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 shutdown_connections 2024-01-19T04:57:22.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.406+0000 7f5f35168640 1 -- 172.21.15.31:0/1697674182 wait complete. 2024-01-19T04:57:22.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:57:22.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.519+0000 7f151ee2b640 1 Processor -- start 2024-01-19T04:57:22.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.519+0000 7f151ee2b640 1 -- start start 2024-01-19T04:57:22.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.519+0000 7f151ee2b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f151806f6c0 0x7f15180cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.519+0000 7f151ee2b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180d0c10 0x7f151806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.519+0000 7f151ee2b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1518162060 con 0x7f15180cfaa0 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.519+0000 7f151ee2b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15180cff40 con 0x7f151806f6c0 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.519+0000 7f151ee2b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15181612a0 con 0x7f15180d0c10 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f1517fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180d0c10 0x7f151806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f151cbc7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f151806f6c0 0x7f15180cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f1517fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180d0c10 0x7f151806a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:44730/0 (socket says 172.21.15.31:44730) 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f151cbc7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f151806f6c0 0x7f15180cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:33832/0 (socket says 172.21.15.31:33832) 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f1517fff640 1 -- 172.21.15.31:0/2310903720 learned_addr learned my addr 172.21.15.31:0/2310903720 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f1517fff640 1 -- 172.21.15.31:0/2310903720 >> v2:172.21.15.137:3300/0 conn(0x7f151806f6c0 msgr2=0x7f15180cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f1517fff640 1 --2- 172.21.15.31:0/2310903720 >> v2:172.21.15.137:3300/0 conn(0x7f151806f6c0 0x7f15180cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f1517fff640 1 -- 172.21.15.31:0/2310903720 >> v1:172.21.15.137:6789/0 conn(0x7f15180cfaa0 legacy=0x7f151815da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.520+0000 7f1517fff640 1 -- 172.21.15.31:0/2310903720 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518161580 con 0x7f15180d0c10 2024-01-19T04:57:22.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f1517fff640 1 --2- 172.21.15.31:0/2310903720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180d0c10 0x7f151806a5b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f150c009870 tx=0x7f150c02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b864ee24f33b3d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f15177fe640 1 -- 172.21.15.31:0/2310903720 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c02f840 con 0x7f15180d0c10 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f15177fe640 1 -- 172.21.15.31:0/2310903720 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c02f9a0 con 0x7f15180d0c10 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f15177fe640 1 -- 172.21.15.31:0/2310903720 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c02fd30 con 0x7f15180d0c10 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f151ee2b640 1 -- 172.21.15.31:0/2310903720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180d0c10 msgr2=0x7f151806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2310903720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180d0c10 0x7f151806a5b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f150c009870 tx=0x7f150c02ba20 comp rx=0 tx=0).stop 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f151ee2b640 1 -- 172.21.15.31:0/2310903720 shutdown_connections 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.521+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2310903720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180d0c10 0x7f151806a5b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.522+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2310903720 >> v2:172.21.15.137:3300/0 conn(0x7f151806f6c0 0x7f15180cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.522+0000 7f151ee2b640 1 -- 172.21.15.31:0/2310903720 >> 172.21.15.31:0/2310903720 conn(0x7f15180d0080 msgr2=0x7f15180d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.522+0000 7f151ee2b640 1 -- 172.21.15.31:0/2310903720 shutdown_connections 2024-01-19T04:57:22.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.522+0000 7f151ee2b640 1 -- 172.21.15.31:0/2310903720 wait complete. 2024-01-19T04:57:22.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.523+0000 7f151ee2b640 1 Processor -- start 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.523+0000 7f151ee2b640 1 -- start start 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.523+0000 7f151ee2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f151806f6c0 0x7f1518155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.523+0000 7f151ee2b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180cfaa0 0x7f151815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151ee2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15180d0c10 0x7f15181580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151ee2b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180b14d0 con 0x7f15180d0c10 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151ee2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180b1a50 con 0x7f151806f6c0 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151ee2b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1518156280 con 0x7f15180cfaa0 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15180d0c10 0x7f15181580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15180d0c10 0x7f15181580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59806/0 (socket says 172.21.15.31:59806) 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 -- 172.21.15.31:0/2081911132 learned_addr learned my addr 172.21.15.31:0/2081911132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151cbc7640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f151806f6c0 0x7f1518155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f1517fff640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180cfaa0 0x7f151815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 -- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180cfaa0 msgr2=0x7f151815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180cfaa0 0x7f151815d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 -- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f151806f6c0 msgr2=0x7f1518155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f151806f6c0 0x7f1518155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 -- 172.21.15.31:0/2081911132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0094d0 con 0x7f15180d0c10 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151cbc7640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f151806f6c0 0x7f1518155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151d3c8640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15180d0c10 0x7f15181580f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f150800c3d0 tx=0x7f150800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f15157fa640 1 -- 172.21.15.31:0/2081911132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15080052d0 con 0x7f15180d0c10 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f15157fa640 1 -- 172.21.15.31:0/2081911132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1508005430 con 0x7f15180d0c10 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15180af9e0 con 0x7f15180d0c10 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f15157fa640 1 -- 172.21.15.31:0/2081911132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508003830 con 0x7f15180d0c10 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.524+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15180b16e0 con 0x7f15180d0c10 2024-01-19T04:57:22.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.525+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15180b1c00 con 0x7f15180d0c10 2024-01-19T04:57:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.526+0000 7f15157fa640 1 -- 172.21.15.31:0/2081911132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f1508003990 con 0x7f15180d0c10 2024-01-19T04:57:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.526+0000 7f15157fa640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f150406aed0 0x7f150406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.526+0000 7f15157fa640 1 -- 172.21.15.31:0/2081911132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1508088c80 con 0x7f15180d0c10 2024-01-19T04:57:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.527+0000 7f151cbc7640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f150406aed0 0x7f150406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.527+0000 7f151cbc7640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f150406aed0 0x7f150406d390 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1510007440 tx=0x7f151000b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.530+0000 7f15157fa640 1 -- 172.21.15.31:0/2081911132 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f15180b1c00 con 0x7f15180d0c10 2024-01-19T04:57:22.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.725+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15180b1c00 con 0x7f15180d0c10 2024-01-19T04:57:22.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.726+0000 7f15157fa640 1 -- 172.21.15.31:0/2081911132 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f15180b1c00 con 0x7f15180d0c10 2024-01-19T04:57:22.725 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f150406aed0 msgr2=0x7f150406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f150406aed0 0x7f150406d390 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1510007440 tx=0x7f151000b040 comp rx=0 tx=0).stop 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15180d0c10 msgr2=0x7f15181580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15180d0c10 0x7f15181580f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f150800c3d0 tx=0x7f150800c890 comp rx=0 tx=0).stop 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 shutdown_connections 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f150406aed0 0x7f150406d390 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15180d0c10 0x7f15181580f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15180cfaa0 0x7f151815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 --2- 172.21.15.31:0/2081911132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f151806f6c0 0x7f1518155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 >> 172.21.15.31:0/2081911132 conn(0x7f15180d0080 msgr2=0x7f151815bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 shutdown_connections 2024-01-19T04:57:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.728+0000 7f151ee2b640 1 -- 172.21.15.31:0/2081911132 wait complete. 2024-01-19T04:57:22.738 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:22.738 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:22.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:22.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.895+0000 7f3cc58da640 1 Processor -- start 2024-01-19T04:57:22.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.895+0000 7f3cc58da640 1 -- start start 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cc58da640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3cc00b1cc0 0x7f3cc00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cc58da640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cc006e9d0 0x7f3cc006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cc58da640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc00b2660 con 0x7f3cc0144070 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cc58da640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc0144450 con 0x7f3cc006e9d0 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cc58da640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc00ced30 con 0x7f3cc00b1cc0 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbf7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cc006e9d0 0x7f3cc006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbeffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3cc0144070 0x7f3cc01445f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48344/0 (socket says 172.21.15.31:48344) 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbeffd640 1 -- 172.21.15.31:0/221858506 learned_addr learned my addr 172.21.15.31:0/221858506 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbe7fc640 1 --2- 172.21.15.31:0/221858506 >> v2:172.21.15.137:3300/0 conn(0x7f3cc00b1cc0 0x7f3cc00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbf7fe640 1 -- 172.21.15.31:0/221858506 >> v2:172.21.15.137:3300/0 conn(0x7f3cc00b1cc0 msgr2=0x7f3cc00b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:22.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbf7fe640 1 --2- 172.21.15.31:0/221858506 >> v2:172.21.15.137:3300/0 conn(0x7f3cc00b1cc0 0x7f3cc00b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbf7fe640 1 -- 172.21.15.31:0/221858506 >> v1:172.21.15.31:6789/0 conn(0x7f3cc0144070 legacy=0x7f3cc01445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbf7fe640 1 -- 172.21.15.31:0/221858506 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc00cf010 con 0x7f3cc006e9d0 2024-01-19T04:57:22.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.896+0000 7f3cbdffb640 1 -- 172.21.15.31:0/221858506 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527067378 0 0) 0x7f3cc00b2660 con 0x7f3cc0144070 2024-01-19T04:57:22.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.897+0000 7f3cbf7fe640 1 --2- 172.21.15.31:0/221858506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc006ee40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0072bd0 tx=0x7f3cb002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c12258d0676b243 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.897+0000 7f3cbdffb640 1 -- 172.21.15.31:0/221858506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb002cc80 con 0x7f3cc006e9d0 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.897+0000 7f3cbdffb640 1 -- 172.21.15.31:0/221858506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb002cde0 con 0x7f3cc006e9d0 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.897+0000 7f3cbdffb640 1 -- 172.21.15.31:0/221858506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0034660 con 0x7f3cc006e9d0 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 -- 172.21.15.31:0/221858506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 msgr2=0x7f3cc006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 --2- 172.21.15.31:0/221858506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc006ee40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0072bd0 tx=0x7f3cb002bed0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 -- 172.21.15.31:0/221858506 shutdown_connections 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 --2- 172.21.15.31:0/221858506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc006ee40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 --2- 172.21.15.31:0/221858506 >> v2:172.21.15.137:3300/0 conn(0x7f3cc00b1cc0 0x7f3cc00b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 -- 172.21.15.31:0/221858506 >> 172.21.15.31:0/221858506 conn(0x7f3cc006d750 msgr2=0x7f3cc006db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 -- 172.21.15.31:0/221858506 shutdown_connections 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.898+0000 7f3cc58da640 1 -- 172.21.15.31:0/221858506 wait complete. 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.899+0000 7f3cc58da640 1 Processor -- start 2024-01-19T04:57:22.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.899+0000 7f3cc58da640 1 -- start start 2024-01-19T04:57:22.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cc58da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc01f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cc58da640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cc00b1cc0 0x7f3cc013baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cc58da640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3cc0144070 0x7f3cc0147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cc58da640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc0146420 con 0x7f3cc006e9d0 2024-01-19T04:57:22.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cc58da640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc013bfe0 con 0x7f3cc00b1cc0 2024-01-19T04:57:22.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cc58da640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc0145810 con 0x7f3cc0144070 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc01f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc01f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59824/0 (socket says 172.21.15.31:59824) 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 -- 172.21.15.31:0/182470665 learned_addr learned my addr 172.21.15.31:0/182470665 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 -- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3cc0144070 msgr2=0x7f3cc0147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3cc0144070 0x7f3cc0147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 -- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cc00b1cc0 msgr2=0x7f3cc013baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cc00b1cc0 0x7f3cc013baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 -- 172.21.15.31:0/182470665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0009a70 con 0x7f3cc006e9d0 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.900+0000 7f3cbeffd640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc01f4fc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c460 tx=0x7f3cb800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.901+0000 7f3cb7fff640 1 -- 172.21.15.31:0/182470665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80074e0 con 0x7f3cc006e9d0 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.901+0000 7f3cb7fff640 1 -- 172.21.15.31:0/182470665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb8007660 con 0x7f3cc006e9d0 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.901+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc0145af0 con 0x7f3cc006e9d0 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.901+0000 7f3cb7fff640 1 -- 172.21.15.31:0/182470665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8004a20 con 0x7f3cc006e9d0 2024-01-19T04:57:22.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.901+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc01460b0 con 0x7f3cc006e9d0 2024-01-19T04:57:22.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.902+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc00696e0 con 0x7f3cc006e9d0 2024-01-19T04:57:22.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.902+0000 7f3cb7fff640 1 -- 172.21.15.31:0/182470665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3cb8004b80 con 0x7f3cc006e9d0 2024-01-19T04:57:22.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.902+0000 7f3cb7fff640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3ca006aed0 0x7f3ca006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:22.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.902+0000 7f3cb7fff640 1 -- 172.21.15.31:0/182470665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cb8088a70 con 0x7f3cc006e9d0 2024-01-19T04:57:22.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.903+0000 7f3cbe7fc640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3ca006aed0 0x7f3ca006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:22.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.903+0000 7f3cbe7fc640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3ca006aed0 0x7f3ca006d390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00b05e0 tx=0x7f3ca8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:22.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:22.907+0000 7f3cb7fff640 1 -- 172.21.15.31:0/182470665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3cb80539d0 con 0x7f3cc006e9d0 2024-01-19T04:57:23.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.085+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cc0147c40 con 0x7f3cc006e9d0 2024-01-19T04:57:23.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.087+0000 7f3cb7fff640 1 -- 172.21.15.31:0/182470665 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f3cb8018020 con 0x7f3cc006e9d0 2024-01-19T04:57:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:23.086 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:23.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.088+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3ca006aed0 msgr2=0x7f3ca006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.088+0000 7f3cc58da640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3ca006aed0 0x7f3ca006d390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00b05e0 tx=0x7f3ca8008040 comp rx=0 tx=0).stop 2024-01-19T04:57:23.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.088+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 msgr2=0x7f3cc01f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.088+0000 7f3cc58da640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc01f4fc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c460 tx=0x7f3cb800c920 comp rx=0 tx=0).stop 2024-01-19T04:57:23.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.088+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 shutdown_connections 2024-01-19T04:57:23.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.089+0000 7f3cc58da640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3ca006aed0 0x7f3ca006d390 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.089+0000 7f3cc58da640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3cc0144070 0x7f3cc0147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.089+0000 7f3cc58da640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cc00b1cc0 0x7f3cc013baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.089+0000 7f3cc58da640 1 --2- 172.21.15.31:0/182470665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cc006e9d0 0x7f3cc01f4fc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.089+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 >> 172.21.15.31:0/182470665 conn(0x7f3cc006d750 msgr2=0x7f3cc00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:23.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.089+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 shutdown_connections 2024-01-19T04:57:23.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.089+0000 7f3cc58da640 1 -- 172.21.15.31:0/182470665 wait complete. 2024-01-19T04:57:23.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:57:10.830975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:57:11.850371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:23.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:23.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.204+0000 7faa27c1e640 1 Processor -- start 2024-01-19T04:57:23.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.204+0000 7faa27c1e640 1 -- start start 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa27c1e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7faa2015d3a0 0x7faa2006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa27c1e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa200cf570 0x7faa20161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa27c1e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa2015d780 0x7faa20161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa27c1e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200cf9f0 con 0x7faa2015d780 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa27c1e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200cfcd0 con 0x7faa2015d3a0 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa27c1e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200cffb0 con 0x7faa200cf570 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa2015d780 0x7faa20161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa2015d780 0x7faa20161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59828/0 (socket says 172.21.15.31:59828) 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 -- 172.21.15.31:0/1031268098 learned_addr learned my addr 172.21.15.31:0/1031268098 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa251b9640 1 --2- 172.21.15.31:0/1031268098 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa200cf570 0x7faa20161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 -- 172.21.15.31:0/1031268098 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa200cf570 msgr2=0x7faa20161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 --2- 172.21.15.31:0/1031268098 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa200cf570 0x7faa20161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 -- 172.21.15.31:0/1031268098 >> v2:172.21.15.137:3300/0 conn(0x7faa2015d3a0 msgr2=0x7faa2006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 --2- 172.21.15.31:0/1031268098 >> v2:172.21.15.137:3300/0 conn(0x7faa2015d3a0 0x7faa2006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa261bb640 1 -- 172.21.15.31:0/1031268098 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa200d0290 con 0x7faa2015d780 2024-01-19T04:57:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.205+0000 7faa251b9640 1 --2- 172.21.15.31:0/1031268098 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa200cf570 0x7faa20161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.206+0000 7faa261bb640 1 --2- 172.21.15.31:0/1031268098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2015d780 0x7faa20161690 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faa2006dd90 tx=0x7faa1c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9ee3f77e9951eb server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.206+0000 7faa249b8640 1 -- 172.21.15.31:0/1031268098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c02ca70 con 0x7faa2015d780 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.206+0000 7faa249b8640 1 -- 172.21.15.31:0/1031268098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c02cbd0 con 0x7faa2015d780 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.206+0000 7faa249b8640 1 -- 172.21.15.31:0/1031268098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0343f0 con 0x7faa2015d780 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 -- 172.21.15.31:0/1031268098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2015d780 msgr2=0x7faa20161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 --2- 172.21.15.31:0/1031268098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2015d780 0x7faa20161690 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faa2006dd90 tx=0x7faa1c02bed0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 -- 172.21.15.31:0/1031268098 shutdown_connections 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 --2- 172.21.15.31:0/1031268098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2015d780 0x7faa20161690 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 --2- 172.21.15.31:0/1031268098 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa200cf570 0x7faa20161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 --2- 172.21.15.31:0/1031268098 >> v2:172.21.15.137:3300/0 conn(0x7faa2015d3a0 0x7faa2006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 -- 172.21.15.31:0/1031268098 >> 172.21.15.31:0/1031268098 conn(0x7faa200d1100 msgr2=0x7faa200d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 -- 172.21.15.31:0/1031268098 shutdown_connections 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.207+0000 7faa27c1e640 1 -- 172.21.15.31:0/1031268098 wait complete. 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.208+0000 7faa27c1e640 1 Processor -- start 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.208+0000 7faa27c1e640 1 -- start start 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa27c1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa200cf570 0x7faa20153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa27c1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa2015d3a0 0x7faa20156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa27c1e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa2015d780 0x7faa201457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa27c1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201f3310 con 0x7faa200cf570 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa27c1e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa20154020 con 0x7faa2015d3a0 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa27c1e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa20146320 con 0x7faa2015d780 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa200cf570 0x7faa20153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa200cf570 0x7faa20153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59840/0 (socket says 172.21.15.31:59840) 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 -- 172.21.15.31:0/3019702974 learned_addr learned my addr 172.21.15.31:0/3019702974 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa261bb640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa2015d780 0x7faa201457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa251b9640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa2015d3a0 0x7faa20156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 -- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa2015d780 msgr2=0x7faa201457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa2015d780 0x7faa201457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 -- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa2015d3a0 msgr2=0x7faa20156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa2015d3a0 0x7faa20156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 -- 172.21.15.31:0/3019702974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009b30 con 0x7faa200cf570 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa259ba640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa200cf570 0x7faa20153ae0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faa1800b4a0 tx=0x7faa1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa167fc640 1 -- 172.21.15.31:0/3019702974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18005a10 con 0x7faa200cf570 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa167fc640 1 -- 172.21.15.31:0/3019702974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa18005b70 con 0x7faa200cf570 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa20146600 con 0x7faa200cf570 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.209+0000 7faa167fc640 1 -- 172.21.15.31:0/3019702974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18010610 con 0x7faa200cf570 2024-01-19T04:57:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.210+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa20155b60 con 0x7faa200cf570 2024-01-19T04:57:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.210+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa20152b00 con 0x7faa200cf570 2024-01-19T04:57:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.211+0000 7faa167fc640 1 -- 172.21.15.31:0/3019702974 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7faa18010770 con 0x7faa200cf570 2024-01-19T04:57:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.211+0000 7faa167fc640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa0406aed0 0x7faa0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.211+0000 7faa167fc640 1 -- 172.21.15.31:0/3019702974 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa18088ec0 con 0x7faa200cf570 2024-01-19T04:57:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.211+0000 7faa251b9640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa0406aed0 0x7faa0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.212+0000 7faa251b9640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa0406aed0 0x7faa0406d390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa200adee0 tx=0x7faa10008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.216+0000 7faa167fc640 1 -- 172.21.15.31:0/3019702974 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7faa18053d80 con 0x7faa200cf570 2024-01-19T04:57:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.392+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa20074900 con 0x7faa200cf570 2024-01-19T04:57:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.392+0000 7faa167fc640 1 -- 172.21.15.31:0/3019702974 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa18014020 con 0x7faa200cf570 2024-01-19T04:57:23.392 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa0406aed0 msgr2=0x7faa0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa0406aed0 0x7faa0406d390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa200adee0 tx=0x7faa10008040 comp rx=0 tx=0).stop 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa200cf570 msgr2=0x7faa20153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa200cf570 0x7faa20153ae0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faa1800b4a0 tx=0x7faa1800b960 comp rx=0 tx=0).stop 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 shutdown_connections 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7faa0406aed0 0x7faa0406d390 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa2015d780 0x7faa201457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa2015d3a0 0x7faa20156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 --2- 172.21.15.31:0/3019702974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa200cf570 0x7faa20153ae0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 >> 172.21.15.31:0/3019702974 conn(0x7faa200d1100 msgr2=0x7faa2015f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 shutdown_connections 2024-01-19T04:57:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.394+0000 7faa27c1e640 1 -- 172.21.15.31:0/3019702974 wait complete. 2024-01-19T04:57:23.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:57:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.508+0000 7fd6f633e640 1 Processor -- start 2024-01-19T04:57:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.508+0000 7fd6f633e640 1 -- start start 2024-01-19T04:57:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6f633e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd6f013e150 0x7fd6f00cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6f633e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00cfc90 con 0x7fd6f015c150 2024-01-19T04:57:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6f633e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f006d030 con 0x7fd6f00cf830 2024-01-19T04:57:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6f633e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f015c590 con 0x7fd6f013e150 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6f48db640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd6f015c150 0x7fd6f00cd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48360/0 (socket says 172.21.15.31:48360) 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6f48db640 1 -- 172.21.15.31:0/3031432537 learned_addr learned my addr 172.21.15.31:0/3031432537 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292588564 0 0) 0x7fd6f00cfc90 con 0x7fd6f015c150 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.509+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d8003520 con 0x7fd6f015c150 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6effff640 1 --2- 172.21.15.31:0/3031432537 >> v2:172.21.15.137:3300/0 conn(0x7fd6f013e150 0x7fd6f00cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1042536578 0 0) 0x7fd6d8003520 con 0x7fd6f015c150 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 >> v2:172.21.15.137:3300/0 conn(0x7fd6f013e150 msgr2=0x7fd6f00cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 --2- 172.21.15.31:0/3031432537 >> v2:172.21.15.137:3300/0 conn(0x7fd6f013e150 0x7fd6f00cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 >> v1:172.21.15.137:6789/0 conn(0x7fd6f00cf830 legacy=0x7fd6f015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f006c720 con 0x7fd6f015c150 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fd6e80019f0 con 0x7fd6f015c150 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 >> v1:172.21.15.31:6789/0 conn(0x7fd6f015c150 legacy=0x7fd6f00cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d8003970 0x7fd6d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6d80047c0 0x7fd6d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f015c590 con 0x7fd6d8005180 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c020 con 0x7fd6d8003970 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c1e0 con 0x7fd6d80047c0 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6f48db640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6effff640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d8003970 0x7fd6d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6d80047c0 0x7fd6d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6f48db640 1 -- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6d80047c0 msgr2=0x7fd6d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6f48db640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6d80047c0 0x7fd6d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6f48db640 1 -- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d8003970 msgr2=0x7fd6d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6f48db640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d8003970 0x7fd6d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6f48db640 1 -- 172.21.15.31:0/3031432537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8003520 con 0x7fd6d8005180 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.510+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6d80047c0 0x7fd6d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.511+0000 7fd6f48db640 1 -- 172.21.15.31:0/3031432537 reap_dead start 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.511+0000 7fd6f48db640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6d800b980 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800a830 tx=0x7fd6d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67de7e4860f14d3f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.511+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80072d0 con 0x7fd6d8005180 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.511+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e802f070 con 0x7fd6d8005180 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.511+0000 7fd6eeffd640 1 -- 172.21.15.31:0/3031432537 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8007690 con 0x7fd6d8005180 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.512+0000 7fd6f633e640 1 -- 172.21.15.31:0/3031432537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 msgr2=0x7fd6d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.512+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6d800b980 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800a830 tx=0x7fd6d80057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.512+0000 7fd6f633e640 1 -- 172.21.15.31:0/3031432537 shutdown_connections 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.512+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3031432537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6d800b980 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.512+0000 7fd6f633e640 1 -- 172.21.15.31:0/3031432537 >> 172.21.15.31:0/3031432537 conn(0x7fd6f006aa00 msgr2=0x7fd6f006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.512+0000 7fd6f633e640 1 -- 172.21.15.31:0/3031432537 shutdown_connections 2024-01-19T04:57:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.512+0000 7fd6f633e640 1 -- 172.21.15.31:0/3031432537 wait complete. 2024-01-19T04:57:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.513+0000 7fd6f633e640 1 Processor -- start 2024-01-19T04:57:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.513+0000 7fd6f633e640 1 -- start start 2024-01-19T04:57:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6f633e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6f0151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6f633e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f013d170 0x7fd6f013b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6f633e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6f013bbc0 0x7fd6f0156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6f633e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0151550 con 0x7fd6d8005180 2024-01-19T04:57:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6f633e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f013c0d0 con 0x7fd6f013d170 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6f633e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f013c3b0 con 0x7fd6f013bbc0 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6f0151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6f0151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59854/0 (socket says 172.21.15.31:59854) 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 -- 172.21.15.31:0/3217600854 learned_addr learned my addr 172.21.15.31:0/3217600854 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f013d170 0x7fd6f013b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6f48db640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6f013bbc0 0x7fd6f0156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 -- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6f013bbc0 msgr2=0x7fd6f0156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6f013bbc0 0x7fd6f0156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 -- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f013d170 msgr2=0x7fd6f013b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f013d170 0x7fd6f013b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 -- 172.21.15.31:0/3217600854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f013c690 con 0x7fd6d8005180 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f013d170 0x7fd6f013b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6effff640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6f0151010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00118e0 tx=0x7fd6e0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.514+0000 7fd6ecff9640 1 -- 172.21.15.31:0/3217600854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000faf0 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.515+0000 7fd6ecff9640 1 -- 172.21.15.31:0/3217600854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e000fc50 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.515+0000 7fd6ecff9640 1 -- 172.21.15.31:0/3217600854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0014440 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.515+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f013c970 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.515+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f0155fc0 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.516+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f0074900 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.516+0000 7fd6ecff9640 1 -- 172.21.15.31:0/3217600854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd6e00145a0 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.516+0000 7fd6ecff9640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd6dc06aed0 0x7fd6dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.517+0000 7fd6ecff9640 1 -- 172.21.15.31:0/3217600854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6e0089f50 con 0x7fd6d8005180 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.517+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd6dc06aed0 0x7fd6dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.517+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd6dc06aed0 0x7fd6dc06d390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4007fd0 tx=0x7fd6e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.521+0000 7fd6ecff9640 1 -- 172.21.15.31:0/3217600854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd6e0054e20 con 0x7fd6d8005180 2024-01-19T04:57:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.701+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6f015b0b0 con 0x7fd6d8005180 2024-01-19T04:57:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.702+0000 7fd6ecff9640 1 -- 172.21.15.31:0/3217600854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6e0058440 con 0x7fd6d8005180 2024-01-19T04:57:23.701 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:57:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.703+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd6dc06aed0 msgr2=0x7fd6dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.703+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd6dc06aed0 0x7fd6dc06d390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4007fd0 tx=0x7fd6e4009040 comp rx=0 tx=0).stop 2024-01-19T04:57:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.703+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 msgr2=0x7fd6f0151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.703+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6f0151010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00118e0 tx=0x7fd6e0011da0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 shutdown_connections 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd6dc06aed0 0x7fd6dc06d390 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd6f013bbc0 0x7fd6f0156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f013d170 0x7fd6f013b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 --2- 172.21.15.31:0/3217600854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d8005180 0x7fd6f0151010 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 >> 172.21.15.31:0/3217600854 conn(0x7fd6f006aa00 msgr2=0x7fd6f0070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 shutdown_connections 2024-01-19T04:57:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.704+0000 7fd6f633e640 1 -- 172.21.15.31:0/3217600854 wait complete. 2024-01-19T04:57:23.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:57:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.819+0000 7fe2f9475640 1 Processor -- start 2024-01-19T04:57:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.819+0000 7fe2f9475640 1 -- start start 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f9475640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe2f415d3a0 0x7fe2f406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f9475640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2f40cf570 0x7fe2f4161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f9475640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f40cf9f0 con 0x7fe2f415d780 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f9475640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f40cfcd0 con 0x7fe2f40cf570 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f9475640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f40cffb0 con 0x7fe2f415d3a0 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f37fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe2f415d780 0x7fe2f40cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48368/0 (socket says 172.21.15.31:48368) 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f37fe640 1 -- 172.21.15.31:0/293820432 learned_addr learned my addr 172.21.15.31:0/293820432 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f27fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2f40cf570 0x7fe2f4161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f2ffd640 1 --2- 172.21.15.31:0/293820432 >> v2:172.21.15.137:3300/0 conn(0x7fe2f415d3a0 0x7fe2f406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f27fc640 1 -- 172.21.15.31:0/293820432 >> v2:172.21.15.137:3300/0 conn(0x7fe2f415d3a0 msgr2=0x7fe2f406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f27fc640 1 --2- 172.21.15.31:0/293820432 >> v2:172.21.15.137:3300/0 conn(0x7fe2f415d3a0 0x7fe2f406c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f27fc640 1 -- 172.21.15.31:0/293820432 >> v1:172.21.15.31:6789/0 conn(0x7fe2f415d780 legacy=0x7fe2f40cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f27fc640 1 -- 172.21.15.31:0/293820432 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f40d0290 con 0x7fe2f40cf570 2024-01-19T04:57:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.820+0000 7fe2f1ffb640 1 -- 172.21.15.31:0/293820432 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590933767 0 0) 0x7fe2f40cf9f0 con 0x7fe2f415d780 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.821+0000 7fe2f27fc640 1 --2- 172.21.15.31:0/293820432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f40cf570 0x7fe2f4161150 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe2f406dd90 tx=0x7fe2e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9ff7aeedfd78bb server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.821+0000 7fe2f1ffb640 1 -- 172.21.15.31:0/293820432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8031830 con 0x7fe2f40cf570 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.821+0000 7fe2f1ffb640 1 -- 172.21.15.31:0/293820432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e8031990 con 0x7fe2f40cf570 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.821+0000 7fe2f1ffb640 1 -- 172.21.15.31:0/293820432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8031d20 con 0x7fe2f40cf570 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.821+0000 7fe2f9475640 1 -- 172.21.15.31:0/293820432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f40cf570 msgr2=0x7fe2f4161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.821+0000 7fe2f9475640 1 --2- 172.21.15.31:0/293820432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f40cf570 0x7fe2f4161150 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe2f406dd90 tx=0x7fe2e802b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.822+0000 7fe2f9475640 1 -- 172.21.15.31:0/293820432 shutdown_connections 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.822+0000 7fe2f9475640 1 --2- 172.21.15.31:0/293820432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f40cf570 0x7fe2f4161150 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.822+0000 7fe2f9475640 1 --2- 172.21.15.31:0/293820432 >> v2:172.21.15.137:3300/0 conn(0x7fe2f415d3a0 0x7fe2f406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.822+0000 7fe2f9475640 1 -- 172.21.15.31:0/293820432 >> 172.21.15.31:0/293820432 conn(0x7fe2f40d1100 msgr2=0x7fe2f40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.822+0000 7fe2f9475640 1 -- 172.21.15.31:0/293820432 shutdown_connections 2024-01-19T04:57:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.822+0000 7fe2f9475640 1 -- 172.21.15.31:0/293820432 wait complete. 2024-01-19T04:57:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.823+0000 7fe2f9475640 1 Processor -- start 2024-01-19T04:57:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.823+0000 7fe2f9475640 1 -- start start 2024-01-19T04:57:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f9475640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2f40cf570 0x7fe2f40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f9475640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f415d3a0 0x7fe2f40b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f9475640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2f415d780 0x7fe2f41f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f9475640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40b1bf0 con 0x7fe2f415d3a0 2024-01-19T04:57:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f9475640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40b1e70 con 0x7fe2f40cf570 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f9475640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40b2150 con 0x7fe2f415d780 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f415d3a0 0x7fe2f40b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f415d3a0 0x7fe2f40b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59862/0 (socket says 172.21.15.31:59862) 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 -- 172.21.15.31:0/2933408260 learned_addr learned my addr 172.21.15.31:0/2933408260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f37fe640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2f415d780 0x7fe2f41f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 -- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2f415d780 msgr2=0x7fe2f41f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2f415d780 0x7fe2f41f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 -- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2f40cf570 msgr2=0x7fe2f40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f2ffd640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2f40cf570 0x7fe2f40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2f40cf570 0x7fe2f40b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 -- 172.21.15.31:0/2933408260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009400 con 0x7fe2f415d3a0 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f37fe640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2f415d780 0x7fe2f41f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2f27fc640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f415d3a0 0x7fe2f40b16b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009890 tx=0x7fe2e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.824+0000 7fe2db7fe640 1 -- 172.21.15.31:0/2933408260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e803ba70 con 0x7fe2f415d3a0 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.825+0000 7fe2db7fe640 1 -- 172.21.15.31:0/2933408260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e803bbf0 con 0x7fe2f415d3a0 2024-01-19T04:57:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.825+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f40b2430 con 0x7fe2f415d3a0 2024-01-19T04:57:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.825+0000 7fe2db7fe640 1 -- 172.21.15.31:0/2933408260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8045400 con 0x7fe2f415d3a0 2024-01-19T04:57:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.825+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f40b29f0 con 0x7fe2f415d3a0 2024-01-19T04:57:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.826+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f4074900 con 0x7fe2f415d3a0 2024-01-19T04:57:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.826+0000 7fe2db7fe640 1 -- 172.21.15.31:0/2933408260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe2e804e440 con 0x7fe2f415d3a0 2024-01-19T04:57:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.826+0000 7fe2db7fe640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2e006aed0 0x7fe2e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.826+0000 7fe2db7fe640 1 -- 172.21.15.31:0/2933408260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2e80ab800 con 0x7fe2f415d3a0 2024-01-19T04:57:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.827+0000 7fe2f2ffd640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2e006aed0 0x7fe2e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.827+0000 7fe2f2ffd640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2e006aed0 0x7fe2e006d390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009770 tx=0x7fe2e4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:23.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:23.831+0000 7fe2db7fe640 1 -- 172.21.15.31:0/2933408260 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe2e8076760 con 0x7fe2f415d3a0 2024-01-19T04:57:24.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.009+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2f4067030 con 0x7fe2f415d3a0 2024-01-19T04:57:24.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.009+0000 7fe2db7fe640 1 -- 172.21.15.31:0/2933408260 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2e8079d80 con 0x7fe2f415d3a0 2024-01-19T04:57:24.009 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2e006aed0 msgr2=0x7fe2e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2e006aed0 0x7fe2e006d390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009770 tx=0x7fe2e4008040 comp rx=0 tx=0).stop 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f415d3a0 msgr2=0x7fe2f40b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f415d3a0 0x7fe2f40b16b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009890 tx=0x7fe2e80046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 shutdown_connections 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe2e006aed0 0x7fe2e006d390 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2f415d780 0x7fe2f41f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2f415d3a0 0x7fe2f40b16b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 --2- 172.21.15.31:0/2933408260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2f40cf570 0x7fe2f40b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 >> 172.21.15.31:0/2933408260 conn(0x7fe2f40d1100 msgr2=0x7fe2f415eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 shutdown_connections 2024-01-19T04:57:24.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.011+0000 7fe2f9475640 1 -- 172.21.15.31:0/2933408260 wait complete. 2024-01-19T04:57:24.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:57:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.124+0000 7fe5ea76f640 1 Processor -- start 2024-01-19T04:57:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.124+0000 7fe5ea76f640 1 -- start start 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5ea76f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 0x7fe5e415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5ea76f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe5e40d0ef0 0x7fe5e40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5ea76f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e415dee0 con 0x7fe5e40d18e0 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5ea76f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e415e1c0 con 0x7fe5e40d0ef0 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5ea76f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415e4a0 con 0x7fe5e40d3e50 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e3fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe5e40d18e0 0x7fe5e40d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48378/0 (socket says 172.21.15.31:48378) 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e3fff640 1 -- 172.21.15.31:0/347508586 learned_addr learned my addr 172.21.15.31:0/347508586 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 0x7fe5e415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e8d0c640 1 --2- 172.21.15.31:0/347508586 >> v2:172.21.15.137:3300/0 conn(0x7fe5e40d0ef0 0x7fe5e40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124425871 0 0) 0x7fe5e415dee0 con 0x7fe5e40d18e0 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5cc003520 con 0x7fe5e40d18e0 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260769280 0 0) 0x7fe5cc003520 con 0x7fe5e40d18e0 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 msgr2=0x7fe5e415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 0x7fe5e415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 >> v2:172.21.15.137:3300/0 conn(0x7fe5e40d0ef0 msgr2=0x7fe5e40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 --2- 172.21.15.31:0/347508586 >> v2:172.21.15.137:3300/0 conn(0x7fe5e40d0ef0 0x7fe5e40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e415e780 con 0x7fe5e40d18e0 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fe5d4001670 con 0x7fe5e40d18e0 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 0x7fe5e415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 >> v1:172.21.15.31:6789/0 conn(0x7fe5e40d18e0 legacy=0x7fe5e40d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc003970 0x7fe5cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.125+0000 7fe5e2ffd640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 0x7fe5cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415e1c0 con 0x7fe5cc0047c0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415e4a0 con 0x7fe5cc003970 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc00b980 con 0x7fe5e40d3e50 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e3fff640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc003970 0x7fe5cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e8d0c640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 0x7fe5cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e37fe640 1 -- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 msgr2=0x7fe5cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40d3e50 0x7fe5cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e37fe640 1 -- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc003970 msgr2=0x7fe5cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc003970 0x7fe5cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.126+0000 7fe5e37fe640 1 -- 172.21.15.31:0/347508586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8009400 con 0x7fe5cc0047c0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009c90 tx=0x7fe5cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39067a3ecc6e511 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d802c8f0 con 0x7fe5cc0047c0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d802ca50 con 0x7fe5cc0047c0 2024-01-19T04:57:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5e2ffd640 1 -- 172.21.15.31:0/347508586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d802ce00 con 0x7fe5cc0047c0 2024-01-19T04:57:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5ea76f640 1 -- 172.21.15.31:0/347508586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 msgr2=0x7fe5cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5ea76f640 1 --2- 172.21.15.31:0/347508586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009c90 tx=0x7fe5cc0057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5e8d0c640 1 -- 172.21.15.31:0/347508586 reap_dead start 2024-01-19T04:57:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5ea76f640 1 -- 172.21.15.31:0/347508586 shutdown_connections 2024-01-19T04:57:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5ea76f640 1 -- 172.21.15.31:0/347508586 >> 172.21.15.31:0/347508586 conn(0x7fe5e409c600 msgr2=0x7fe5e40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5ea76f640 1 -- 172.21.15.31:0/347508586 shutdown_connections 2024-01-19T04:57:24.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.127+0000 7fe5ea76f640 1 -- 172.21.15.31:0/347508586 wait complete. 2024-01-19T04:57:24.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.128+0000 7fe5ea76f640 1 Processor -- start 2024-01-19T04:57:24.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5ea76f640 1 -- start start 2024-01-19T04:57:24.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5ea76f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5e40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5ea76f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e40b4720 0x7fe5e40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5ea76f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40b4b80 0x7fe5e40ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5ea76f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40ca3f0 con 0x7fe5cc0047c0 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5ea76f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40ca6d0 con 0x7fe5e40b4720 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5ea76f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40cbe10 con 0x7fe5e40b4b80 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5e40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5e40b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59880/0 (socket says 172.21.15.31:59880) 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.129+0000 7fe5e3fff640 1 -- 172.21.15.31:0/3600026643 learned_addr learned my addr 172.21.15.31:0/3600026643 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e3fff640 1 -- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40b4b80 msgr2=0x7fe5e40ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e3fff640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40b4b80 0x7fe5e40ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e3fff640 1 -- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e40b4720 msgr2=0x7fe5e40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e3fff640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e40b4720 0x7fe5e40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e3fff640 1 -- 172.21.15.31:0/3600026643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40b50b0 con 0x7fe5cc0047c0 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e3fff640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5e40b41e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4007340 tx=0x7fe5d4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e0ff9640 1 -- 172.21.15.31:0/3600026643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400f840 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e0ff9640 1 -- 172.21.15.31:0/3600026643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d400f9a0 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e40b5390 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5e0ff9640 1 -- 172.21.15.31:0/3600026643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400fd50 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.130+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e4152d00 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.131+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4074900 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.132+0000 7fe5e0ff9640 1 -- 172.21.15.31:0/3600026643 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe5d401d460 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.132+0000 7fe5e0ff9640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.132+0000 7fe5e0ff9640 1 -- 172.21.15.31:0/3600026643 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5d4089770 con 0x7fe5cc0047c0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.132+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.133+0000 7fe5e37fe640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe5d006aed0 0x7fe5d006d390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8036000 tx=0x7fe5d802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.136+0000 7fe5e0ff9640 1 -- 172.21.15.31:0/3600026643 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe5d4054790 con 0x7fe5cc0047c0 2024-01-19T04:57:24.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.325+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe5e40b15b0 con 0x7fe5cc0047c0 2024-01-19T04:57:24.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.326+0000 7fe5e0ff9640 1 -- 172.21.15.31:0/3600026643 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe5d4057db0 con 0x7fe5cc0047c0 2024-01-19T04:57:24.325 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:57:24.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe5d006aed0 msgr2=0x7fe5d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe5d006aed0 0x7fe5d006d390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8036000 tx=0x7fe5d802e000 comp rx=0 tx=0).stop 2024-01-19T04:57:24.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 msgr2=0x7fe5e40b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5e40b41e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4007340 tx=0x7fe5d4007800 comp rx=0 tx=0).stop 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 shutdown_connections 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5e40b4b80 0x7fe5e40ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e40b4720 0x7fe5e40cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 --2- 172.21.15.31:0/3600026643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5cc0047c0 0x7fe5e40b41e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 >> 172.21.15.31:0/3600026643 conn(0x7fe5e409c600 msgr2=0x7fe5e4160380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 shutdown_connections 2024-01-19T04:57:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.328+0000 7fe5ea76f640 1 -- 172.21.15.31:0/3600026643 wait complete. 2024-01-19T04:57:24.338 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:57:24.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:24.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:24.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[56381]: fuse finished with error 0 and tester_r 0 2024-01-19T04:57:24.364 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:57:24.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:57:24.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:24.464 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:57:24.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:57:24.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.621+0000 7efe3b753640 1 Processor -- start 2024-01-19T04:57:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.621+0000 7efe3b753640 1 -- start start 2024-01-19T04:57:24.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe3b753640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efe340cfd10 0x7efe340d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe3b753640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe340d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe3b753640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe340d06b0 con 0x7efe34160bb0 2024-01-19T04:57:24.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe3b753640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe340d18c0 con 0x7efe340cfd10 2024-01-19T04:57:24.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe3b753640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe340d1b90 con 0x7efe340d0ef0 2024-01-19T04:57:24.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe394ef640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efe34160bb0 0x7efe340b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48392/0 (socket says 172.21.15.31:48392) 2024-01-19T04:57:24.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe394ef640 1 -- 172.21.15.31:0/3910756277 learned_addr learned my addr 172.21.15.31:0/3910756277 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:24.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509492481 0 0) 0x7efe340d06b0 con 0x7efe34160bb0 2024-01-19T04:57:24.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe1c003520 con 0x7efe34160bb0 2024-01-19T04:57:24.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe38cee640 1 --2- 172.21.15.31:0/3910756277 >> v2:172.21.15.137:3300/0 conn(0x7efe340cfd10 0x7efe340d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe39cf0640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe340d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000230246 0 0) 0x7efe1c003520 con 0x7efe34160bb0 2024-01-19T04:57:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 msgr2=0x7efe340d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe340d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 >> v2:172.21.15.137:3300/0 conn(0x7efe340cfd10 msgr2=0x7efe340d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 --2- 172.21.15.31:0/3910756277 >> v2:172.21.15.137:3300/0 conn(0x7efe340cfd10 0x7efe340d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.622+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340d1e70 con 0x7efe34160bb0 2024-01-19T04:57:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7efe2c001670 con 0x7efe34160bb0 2024-01-19T04:57:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe39cf0640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe340d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 >> v1:172.21.15.31:6789/0 conn(0x7efe34160bb0 legacy=0x7efe340b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c003970 0x7efe1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe1c0047c0 0x7efe1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340d18c0 con 0x7efe1c003970 2024-01-19T04:57:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340d1b90 con 0x7efe1c0047c0 2024-01-19T04:57:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c00b980 con 0x7efe340d0ef0 2024-01-19T04:57:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe394ef640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c003970 0x7efe1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe39cf0640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe38cee640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe1c0047c0 0x7efe1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe394ef640 1 -- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 msgr2=0x7efe1c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe394ef640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe1c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe394ef640 1 -- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe1c0047c0 msgr2=0x7efe1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:24.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe394ef640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe1c0047c0 0x7efe1c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe394ef640 1 -- 172.21.15.31:0/3910756277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c003520 con 0x7efe1c003970 2024-01-19T04:57:24.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe39cf0640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe340d0ef0 0x7efe1c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:24.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.623+0000 7efe38cee640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe1c0047c0 0x7efe1c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:24.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe394ef640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c003970 0x7efe1c003d90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c270 tx=0x7efe1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255c8c52a88f70f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:24.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c02cae0 con 0x7efe1c003970 2024-01-19T04:57:24.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c02cc40 con 0x7efe1c003970 2024-01-19T04:57:24.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe33fff640 1 -- 172.21.15.31:0/3910756277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0058d0 con 0x7efe1c003970 2024-01-19T04:57:24.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe3b753640 1 -- 172.21.15.31:0/3910756277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c003970 msgr2=0x7efe1c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe3b753640 1 --2- 172.21.15.31:0/3910756277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c003970 0x7efe1c003d90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c270 tx=0x7efe1c0057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe39cf0640 1 -- 172.21.15.31:0/3910756277 reap_dead start 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe3b753640 1 -- 172.21.15.31:0/3910756277 shutdown_connections 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe3b753640 1 -- 172.21.15.31:0/3910756277 >> 172.21.15.31:0/3910756277 conn(0x7efe3409c600 msgr2=0x7efe340d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe3b753640 1 -- 172.21.15.31:0/3910756277 shutdown_connections 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.624+0000 7efe3b753640 1 -- 172.21.15.31:0/3910756277 wait complete. 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 Processor -- start 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 -- start start 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe1c003970 0x7efe340b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe340b7110 0x7efe3415a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe341f36c0 0x7efe341f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3415b9e0 con 0x7efe341f36c0 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340b6d60 con 0x7efe340b7110 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe3b753640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3415af10 con 0x7efe1c003970 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe39cf0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe341f36c0 0x7efe341f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe39cf0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe341f36c0 0x7efe341f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59908/0 (socket says 172.21.15.31:59908) 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.626+0000 7efe39cf0640 1 -- 172.21.15.31:0/852926963 learned_addr learned my addr 172.21.15.31:0/852926963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:24.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe394ef640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe1c003970 0x7efe340b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe38cee640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe340b7110 0x7efe3415a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe39cf0640 1 -- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe1c003970 msgr2=0x7efe340b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe39cf0640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe1c003970 0x7efe340b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe39cf0640 1 -- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe340b7110 msgr2=0x7efe3415a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe39cf0640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe340b7110 0x7efe3415a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe39cf0640 1 -- 172.21.15.31:0/852926963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340b7660 con 0x7efe341f36c0 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe39cf0640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe341f36c0 0x7efe341f3b40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efe3406b260 tx=0x7efe24004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe31ffb640 1 -- 172.21.15.31:0/852926963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24007c70 con 0x7efe341f36c0 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe31ffb640 1 -- 172.21.15.31:0/852926963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2400f050 con 0x7efe341f36c0 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe340b7940 con 0x7efe341f36c0 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe31ffb640 1 -- 172.21.15.31:0/852926963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24014410 con 0x7efe341f36c0 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.627+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe340b7f00 con 0x7efe341f36c0 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.628+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe340a10e0 con 0x7efe341f36c0 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.629+0000 7efe31ffb640 1 -- 172.21.15.31:0/852926963 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efe24014570 con 0x7efe341f36c0 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.629+0000 7efe31ffb640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe2006af00 0x7efe2006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.629+0000 7efe31ffb640 1 -- 172.21.15.31:0/852926963 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe24089b80 con 0x7efe341f36c0 2024-01-19T04:57:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.629+0000 7efe394ef640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe2006af00 0x7efe2006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.630+0000 7efe394ef640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe2006af00 0x7efe2006d3c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe2c034000 tx=0x7efe2c037040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.633+0000 7efe31ffb640 1 -- 172.21.15.31:0/852926963 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efe240549b0 con 0x7efe341f36c0 2024-01-19T04:57:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.850+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efe34067b50 con 0x7efe341f36c0 2024-01-19T04:57:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.850+0000 7efe31ffb640 1 -- 172.21.15.31:0/852926963 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7efe24089e60 con 0x7efe341f36c0 2024-01-19T04:57:24.850 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe2006af00 msgr2=0x7efe2006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe2006af00 0x7efe2006d3c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe2c034000 tx=0x7efe2c037040 comp rx=0 tx=0).stop 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe341f36c0 msgr2=0x7efe341f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe341f36c0 0x7efe341f3b40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efe3406b260 tx=0x7efe24004770 comp rx=0 tx=0).stop 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 shutdown_connections 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efe2006af00 0x7efe2006d3c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe341f36c0 0x7efe341f3b40 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe340b7110 0x7efe3415a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 --2- 172.21.15.31:0/852926963 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe1c003970 0x7efe340b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:24.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 >> 172.21.15.31:0/852926963 conn(0x7efe3409c600 msgr2=0x7efe3415d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:24.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 shutdown_connections 2024-01-19T04:57:24.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:24.852+0000 7efe3b753640 1 -- 172.21.15.31:0/852926963 wait complete. 2024-01-19T04:57:24.862 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:57:24.862 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:57:24.862 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:57:24.863 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-19T04:57:24.863 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:24.863 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:57:24.863 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:57:24.863 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:57:24.863 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:24.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:24.863 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:57:24.878 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42648sec preferred_lft 42648sec 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c57:fcff:feba:867a/64 scope link 2024-01-19T04:57:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::98a9:56ff:fef7:e409/64 scope link 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::90b1:87ff:fefd:d7a/64 scope link 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:24.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:24.881 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:57:24.944 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:57:24.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:24.946 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:57:25.001 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:57:25.001 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:57:25.001 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:57:25.002 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:57:25.002 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:57:25.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:25.002 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:57:25.002 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:57:25.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:57:25.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:57:25.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:57:25.002 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:57:25.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:25.096 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:57:25.112 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-19T04:57:25.112 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:57:25.112 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:25.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:25.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-01-19T04:57:25.179 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-19T04:57:25.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:57:25.179 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:25.233 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:25.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:57:25.234 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:25.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:57:25.355 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:57:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:57:25.412 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:25.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:25.413 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:57:25.465 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:57:25.466 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-19T04:57:25.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:57:25.534 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:57:25.589 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:57:25.644 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:57:25.645 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:57:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:25.647 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:25.647 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:25.647 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:25.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:25.648 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:57:25.648 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-01-19T04:57:25.593+0000 7ffa70a45480 -1 init, newargv = 0x558b282926e0 newargc=15 2024-01-19T04:57:25.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[57345]: starting ceph client 2024-01-19T04:57:25.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[57345]: starting fuse 2024-01-19T04:57:25.713 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:57:25.713 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:57:25.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:57:25.713 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:57:25.714 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-01-19T04:57:25.715 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:57:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.57345.asok 2024-01-19T04:57:25.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:57:25.845 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57345.asok 2024-01-19T04:57:25.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:25.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57345.asok status 2024-01-19T04:57:25.982 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "57345", 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6174, 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 6174 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 65089333 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 65089333 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.6174 192.168.0.1:0/65089333", 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/65089333", 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 22, 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 50, 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:57:25.985 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:25.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:25.992 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:57:25.992 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:26.007 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:57:26.007 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:26.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:26.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:26.074 DEBUG:teuthology.orchestra.run.smithi031:> stdin-killer -- python3 -c ' 2024-01-19T04:57:26.074 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:57:26.075 DEBUG:teuthology.orchestra.run.smithi031:> import errno 2024-01-19T04:57:26.075 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:26.075 DEBUG:teuthology.orchestra.run.smithi031:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-01-19T04:57:26.075 DEBUG:teuthology.orchestra.run.smithi031:> try: 2024-01-19T04:57:26.075 DEBUG:teuthology.orchestra.run.smithi031:> if False: 2024-01-19T04:57:26.076 DEBUG:teuthology.orchestra.run.smithi031:> ret = os.read(fd, 1024) 2024-01-19T04:57:26.076 DEBUG:teuthology.orchestra.run.smithi031:> else: 2024-01-19T04:57:26.076 DEBUG:teuthology.orchestra.run.smithi031:> os.write(fd, b'"'"'content'"'"') 2024-01-19T04:57:26.076 DEBUG:teuthology.orchestra.run.smithi031:> except OSError as e: 2024-01-19T04:57:26.076 DEBUG:teuthology.orchestra.run.smithi031:> if e.errno != errno.EPERM: 2024-01-19T04:57:26.077 DEBUG:teuthology.orchestra.run.smithi031:> raise 2024-01-19T04:57:26.077 DEBUG:teuthology.orchestra.run.smithi031:> else: 2024-01-19T04:57:26.077 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("client does not check permission of data pool") 2024-01-19T04:57:26.077 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:57:26.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:57:26.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2024-01-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.340+0000 7f9df1d94640 1 Processor -- start 2024-01-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.340+0000 7f9df1d94640 1 -- start start 2024-01-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.340+0000 7f9df1d94640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec15d720 0x7f9dec0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.340+0000 7f9df1d94640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9dec0d1c20 0x7f9dec0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.340+0000 7f9df1d94640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec0cf160 con 0x7f9dec0d3ea0 2024-01-19T04:57:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.340+0000 7f9df1d94640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec0ce8c0 con 0x7f9dec0d1c20 2024-01-19T04:57:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.340+0000 7f9df1d94640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0ceba0 con 0x7f9dec15d720 2024-01-19T04:57:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec15d720 0x7f9dec0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9debfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9dec0d1c20 0x7f9dec0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec15d720 0x7f9dec0d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:44844/0 (socket says 172.21.15.31:44844) 2024-01-19T04:57:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9debfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9dec0d1c20 0x7f9dec0b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:33986/0 (socket says 172.21.15.31:33986) 2024-01-19T04:57:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9deb7fe640 1 -- 172.21.15.31:0/3337978825 learned_addr learned my addr 172.21.15.31:0/3337978825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9debfff640 1 -- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec15d720 msgr2=0x7f9dec0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9debfff640 1 --2- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec15d720 0x7f9dec0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9debfff640 1 -- 172.21.15.31:0/3337978825 >> v1:172.21.15.137:6789/0 conn(0x7f9dec0d3ea0 legacy=0x7f9dec0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9debfff640 1 -- 172.21.15.31:0/3337978825 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0cee80 con 0x7f9dec0d1c20 2024-01-19T04:57:26.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.341+0000 7f9deb7fe640 1 --2- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec15d720 0x7f9dec0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.342+0000 7f9debfff640 1 --2- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d1c20 0x7f9dec0b3000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009d90 tx=0x7f9de402bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e816b1b4b38cbdd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.342+0000 7f9deaffd640 1 -- 172.21.15.31:0/3337978825 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de402cb30 con 0x7f9dec0d1c20 2024-01-19T04:57:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.342+0000 7f9deaffd640 1 -- 172.21.15.31:0/3337978825 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de402cc90 con 0x7f9dec0d1c20 2024-01-19T04:57:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.342+0000 7f9deaffd640 1 -- 172.21.15.31:0/3337978825 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4034440 con 0x7f9dec0d1c20 2024-01-19T04:57:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 -- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d1c20 msgr2=0x7f9dec0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 --2- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d1c20 0x7f9dec0b3000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009d90 tx=0x7f9de402bd80 comp rx=0 tx=0).stop 2024-01-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 -- 172.21.15.31:0/3337978825 shutdown_connections 2024-01-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 --2- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d1c20 0x7f9dec0b3000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 --2- 172.21.15.31:0/3337978825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec15d720 0x7f9dec0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 -- 172.21.15.31:0/3337978825 >> 172.21.15.31:0/3337978825 conn(0x7f9dec0d3310 msgr2=0x7f9dec0d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 -- 172.21.15.31:0/3337978825 shutdown_connections 2024-01-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.343+0000 7f9df1d94640 1 -- 172.21.15.31:0/3337978825 wait complete. 2024-01-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.344+0000 7f9df1d94640 1 Processor -- start 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.344+0000 7f9df1d94640 1 -- start start 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9df1d94640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec0d1c20 0x7f9dec0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9df1d94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d3ea0 0x7f9dec1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9df1d94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dec15d720 0x7f9dec0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9df1d94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec152de0 con 0x7f9dec15d720 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9df1d94640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0cb330 con 0x7f9dec0d3ea0 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9df1d94640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0cb5d0 con 0x7f9dec0d1c20 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dec15d720 0x7f9dec0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dec15d720 0x7f9dec0cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59918/0 (socket says 172.21.15.31:59918) 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 -- 172.21.15.31:0/547793396 learned_addr learned my addr 172.21.15.31:0/547793396 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:26.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9deb7fe640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec0d1c20 0x7f9dec0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9de3fff640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d3ea0 0x7f9dec1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 -- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec0d1c20 msgr2=0x7f9dec0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec0d1c20 0x7f9dec0ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 -- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d3ea0 msgr2=0x7f9dec1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d3ea0 0x7f9dec1528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 -- 172.21.15.31:0/547793396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0094d0 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9debfff640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dec15d720 0x7f9dec0cadf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009d90 tx=0x7f9de4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9de8ff9640 1 -- 172.21.15.31:0/547793396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4040450 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9de8ff9640 1 -- 172.21.15.31:0/547793396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de40405d0 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.345+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de4009a70 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.346+0000 7f9de8ff9640 1 -- 172.21.15.31:0/547793396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4040980 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.346+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec0cbb90 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.346+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec074900 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.347+0000 7f9de8ff9640 1 -- 172.21.15.31:0/547793396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9de4040ae0 con 0x7f9dec15d720 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.347+0000 7f9de8ff9640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9dcc06aed0 0x7f9dcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.347+0000 7f9de8ff9640 1 -- 172.21.15.31:0/547793396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9de40ac130 con 0x7f9dec15d720 2024-01-19T04:57:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.348+0000 7f9deb7fe640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9dcc06aed0 0x7f9dcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.348+0000 7f9deb7fe640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9dcc06aed0 0x7f9dcc06d390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc006ca0 tx=0x7f9ddc009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.351+0000 7f9de8ff9640 1 -- 172.21.15.31:0/547793396 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9de40770a0 con 0x7f9dec15d720 2024-01-19T04:57:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.600+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7f9dec0d0120 con 0x7f9dec15d720 2024-01-19T04:57:26.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.602+0000 7f9de8ff9640 1 -- 172.21.15.31:0/547793396 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 183+0+140 (secure 0 0 0) 0x7f9de407a6c0 con 0x7f9dec15d720 2024-01-19T04:57:26.602 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:57:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:57:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2024-01-19T04:57:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:57:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow w pool=cephfs_data" 2024-01-19T04:57:26.603 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:57:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9dcc06aed0 msgr2=0x7f9dcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9dcc06aed0 0x7f9dcc06d390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc006ca0 tx=0x7f9ddc009240 comp rx=0 tx=0).stop 2024-01-19T04:57:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dec15d720 msgr2=0x7f9dec0cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dec15d720 0x7f9dec0cadf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009d90 tx=0x7f9de4004a40 comp rx=0 tx=0).stop 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 shutdown_connections 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9dcc06aed0 0x7f9dcc06d390 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dec15d720 0x7f9dec0cadf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dec0d3ea0 0x7f9dec1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 --2- 172.21.15.31:0/547793396 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9dec0d1c20 0x7f9dec0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 >> 172.21.15.31:0/547793396 conn(0x7f9dec0d3310 msgr2=0x7f9dec06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 shutdown_connections 2024-01-19T04:57:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.605+0000 7f9df1d94640 1 -- 172.21.15.31:0/547793396 wait complete. 2024-01-19T04:57:26.615 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:26.615 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:26.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.771+0000 7fb223c6b640 1 Processor -- start 2024-01-19T04:57:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.772+0000 7fb223c6b640 1 -- start start 2024-01-19T04:57:26.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.772+0000 7fb223c6b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 0x7fb21c0b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.772+0000 7fb223c6b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c160df0 con 0x7fb21c160a10 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.772+0000 7fb223c6b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c0b7300 con 0x7fb21c0d2e50 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.772+0000 7fb223c6b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c0d1db0 con 0x7fb21c0b5410 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.772+0000 7fb221a07640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb21c160a10 0x7fb21c0d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48394/0 (socket says 172.21.15.31:48394) 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.772+0000 7fb221a07640 1 -- 172.21.15.31:0/573847132 learned_addr learned my addr 172.21.15.31:0/573847132 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb222208640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 0x7fb21c0b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005834812 0 0) 0x7fb21c160df0 con 0x7fb21c160a10 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb208003520 con 0x7fb21c160a10 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3617435045 0 0) 0x7fb208003520 con 0x7fb21c160a10 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 msgr2=0x7fb21c0b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 0x7fb21c0b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 >> v1:172.21.15.137:6789/0 conn(0x7fb21c0d2e50 legacy=0x7fb21c0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c0d2090 con 0x7fb21c160a10 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fb218001670 con 0x7fb21c160a10 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 >> v1:172.21.15.31:6789/0 conn(0x7fb21c160a10 legacy=0x7fb21c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb208003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb208004350 0x7fb208004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 0x7fb208005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c0d1db0 con 0x7fb2080039a0 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20800b9b0 con 0x7fb208004350 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20800bb70 con 0x7fb21c0b5410 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb221a07640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb208003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb222208640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 0x7fb208005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb221206640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb208004350 0x7fb208004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb221a07640 1 -- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 msgr2=0x7fb208005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:26.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.773+0000 7fb221a07640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 0x7fb208005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb221a07640 1 -- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb208004350 msgr2=0x7fb208004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb221a07640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb208004350 0x7fb208004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb221a07640 1 -- 172.21.15.31:0/573847132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208003520 con 0x7fb2080039a0 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb222208640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c0b5410 0x7fb208005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb221a07640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb208003d80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb21800c240 tx=0x7fb2080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698c08262aa7817e server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218005ad0 con 0x7fb2080039a0 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218005c30 con 0x7fb2080039a0 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.774+0000 7fb220a05640 1 -- 172.21.15.31:0/573847132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21802c8d0 con 0x7fb2080039a0 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.775+0000 7fb223c6b640 1 -- 172.21.15.31:0/573847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 msgr2=0x7fb208003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.775+0000 7fb223c6b640 1 --2- 172.21.15.31:0/573847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb208003d80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb21800c240 tx=0x7fb2080057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.775+0000 7fb222208640 1 -- 172.21.15.31:0/573847132 reap_dead start 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.775+0000 7fb223c6b640 1 -- 172.21.15.31:0/573847132 shutdown_connections 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.775+0000 7fb223c6b640 1 -- 172.21.15.31:0/573847132 >> 172.21.15.31:0/573847132 conn(0x7fb21c072310 msgr2=0x7fb21c0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:26.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.775+0000 7fb223c6b640 1 -- 172.21.15.31:0/573847132 shutdown_connections 2024-01-19T04:57:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.775+0000 7fb223c6b640 1 -- 172.21.15.31:0/573847132 wait complete. 2024-01-19T04:57:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.776+0000 7fb223c6b640 1 Processor -- start 2024-01-19T04:57:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.776+0000 7fb223c6b640 1 -- start start 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb223c6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb21c1e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb223c6b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21c1e8ca0 0x7fb21c1e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb223c6b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c1e6830 0x7fb21c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb223c6b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1e6d20 con 0x7fb2080039a0 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb223c6b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1e7000 con 0x7fb21c1e8ca0 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb223c6b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1e72e0 con 0x7fb21c1e6830 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb21c1e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb21c1e7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59934/0 (socket says 172.21.15.31:59934) 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 -- 172.21.15.31:0/1178324769 learned_addr learned my addr 172.21.15.31:0/1178324769 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb222208640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c1e6830 0x7fb21c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 -- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c1e6830 msgr2=0x7fb21c14e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c1e6830 0x7fb21c14e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 -- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21c1e8ca0 msgr2=0x7fb21c1e62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21c1e8ca0 0x7fb21c1e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 -- 172.21.15.31:0/1178324769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c1e75c0 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.777+0000 7fb221a07640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb21c1e7dc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb218000f60 tx=0x7fb21c0d14b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.778+0000 7fb2127fc640 1 -- 172.21.15.31:0/1178324769 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218039070 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.778+0000 7fb2127fc640 1 -- 172.21.15.31:0/1178324769 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21802cce0 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.778+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c1e78a0 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.778+0000 7fb2127fc640 1 -- 172.21.15.31:0/1178324769 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180072b0 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.778+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c1e9100 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.779+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21c1e93b0 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.779+0000 7fb2127fc640 1 -- 172.21.15.31:0/1178324769 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb218032070 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.780+0000 7fb2127fc640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb20006aed0 0x7fb20006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.780+0000 7fb2127fc640 1 -- 172.21.15.31:0/1178324769 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb21803c040 con 0x7fb2080039a0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.780+0000 7fb221206640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb20006aed0 0x7fb20006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:26.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.780+0000 7fb221206640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb20006aed0 0x7fb20006d390 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb21c15b760 tx=0x7fb20c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:26.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.784+0000 7fb2127fc640 1 -- 172.21.15.31:0/1178324769 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb2180757a0 con 0x7fb2080039a0 2024-01-19T04:57:26.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.961+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb21c0b5230 con 0x7fb2080039a0 2024-01-19T04:57:26.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.962+0000 7fb2127fc640 1 -- 172.21.15.31:0/1178324769 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fb218078dc0 con 0x7fb2080039a0 2024-01-19T04:57:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:26.962 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb20006aed0 msgr2=0x7fb20006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb20006aed0 0x7fb20006d390 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb21c15b760 tx=0x7fb20c009040 comp rx=0 tx=0).stop 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 msgr2=0x7fb21c1e7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb21c1e7dc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb218000f60 tx=0x7fb21c0d14b0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 shutdown_connections 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb20006aed0 0x7fb20006d390 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb21c1e6830 0x7fb21c14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb21c1e8ca0 0x7fb21c1e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 --2- 172.21.15.31:0/1178324769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2080039a0 0x7fb21c1e7dc0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 >> 172.21.15.31:0/1178324769 conn(0x7fb21c072310 msgr2=0x7fb21c0b62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 shutdown_connections 2024-01-19T04:57:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:26.964+0000 7fb223c6b640 1 -- 172.21.15.31:0/1178324769 wait complete. 2024-01-19T04:57:26.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:57:10.830975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:57:11.850371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:26.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.079+0000 7f2d109c9640 1 Processor -- start 2024-01-19T04:57:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.079+0000 7f2d109c9640 1 -- start start 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d109c9640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2d0c15b4b0 0x7f2d0c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d109c9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0c9900 0x7f2d0c071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d109c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d0c0720a0 0x7f2d0c06c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d109c9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c072530 con 0x7f2d0c0720a0 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d109c9640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c06e5d0 con 0x7f2d0c15b4b0 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d109c9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c15b890 con 0x7f2d0c0c9900 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d0c0720a0 0x7f2d0c06c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d0c0720a0 0x7f2d0c06c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59940/0 (socket says 172.21.15.31:59940) 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 -- 172.21.15.31:0/1245292571 learned_addr learned my addr 172.21.15.31:0/1245292571 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0a59c640 1 --2- 172.21.15.31:0/1245292571 >> v2:172.21.15.137:3300/0 conn(0x7f2d0c15b4b0 0x7f2d0c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d09d9b640 1 --2- 172.21.15.31:0/1245292571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0c9900 0x7f2d0c071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 -- 172.21.15.31:0/1245292571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0c9900 msgr2=0x7f2d0c071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 --2- 172.21.15.31:0/1245292571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0c9900 0x7f2d0c071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 -- 172.21.15.31:0/1245292571 >> v2:172.21.15.137:3300/0 conn(0x7f2d0c15b4b0 msgr2=0x7f2d0c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 --2- 172.21.15.31:0/1245292571 >> v2:172.21.15.137:3300/0 conn(0x7f2d0c15b4b0 0x7f2d0c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.080+0000 7f2d0ad9d640 1 -- 172.21.15.31:0/1245292571 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c06d9c0 con 0x7f2d0c0720a0 2024-01-19T04:57:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.081+0000 7f2d0ad9d640 1 --2- 172.21.15.31:0/1245292571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0720a0 0x7f2d0c06c810 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfbd50294cf9e6b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.081+0000 7f2d0959a640 1 -- 172.21.15.31:0/1245292571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0402ca70 con 0x7f2d0c0720a0 2024-01-19T04:57:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.081+0000 7f2d0959a640 1 -- 172.21.15.31:0/1245292571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0402cbd0 con 0x7f2d0c0720a0 2024-01-19T04:57:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.081+0000 7f2d0959a640 1 -- 172.21.15.31:0/1245292571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040343f0 con 0x7f2d0c0720a0 2024-01-19T04:57:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.081+0000 7f2d109c9640 1 -- 172.21.15.31:0/1245292571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0720a0 msgr2=0x7f2d0c06c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.081+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1245292571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0720a0 0x7f2d0c06c810 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0402bed0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.082+0000 7f2d109c9640 1 -- 172.21.15.31:0/1245292571 shutdown_connections 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.082+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1245292571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0720a0 0x7f2d0c06c810 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.082+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1245292571 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0c9900 0x7f2d0c071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.082+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1245292571 >> v2:172.21.15.137:3300/0 conn(0x7f2d0c15b4b0 0x7f2d0c0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.082+0000 7f2d109c9640 1 -- 172.21.15.31:0/1245292571 >> 172.21.15.31:0/1245292571 conn(0x7f2d0c0b0da0 msgr2=0x7f2d0c0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.082+0000 7f2d109c9640 1 -- 172.21.15.31:0/1245292571 shutdown_connections 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.082+0000 7f2d109c9640 1 -- 172.21.15.31:0/1245292571 wait complete. 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.083+0000 7f2d109c9640 1 Processor -- start 2024-01-19T04:57:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.083+0000 7f2d109c9640 1 -- start start 2024-01-19T04:57:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d109c9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0720a0 0x7f2d0c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d109c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0c9900 0x7f2d0c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d109c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c15b4b0 0x7f2d0c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d109c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1f19f0 con 0x7f2d0c0c9900 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d109c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1f0e00 con 0x7f2d0c15b4b0 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d109c9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1f10c0 con 0x7f2d0c0720a0 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0c9900 0x7f2d0c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0c9900 0x7f2d0c0c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59944/0 (socket says 172.21.15.31:59944) 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 -- 172.21.15.31:0/1630072763 learned_addr learned my addr 172.21.15.31:0/1630072763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d0a59c640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0720a0 0x7f2d0c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d0ad9d640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c15b4b0 0x7f2d0c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 -- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0720a0 msgr2=0x7f2d0c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0720a0 0x7f2d0c1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 -- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c15b4b0 msgr2=0x7f2d0c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c15b4b0 0x7f2d0c0c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 -- 172.21.15.31:0/1630072763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d04009b30 con 0x7f2d0c0c9900 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.084+0000 7f2d09d9b640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0c9900 0x7f2d0c0c8780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000d800 tx=0x7f2d0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.085+0000 7f2cfaffd640 1 -- 172.21.15.31:0/1630072763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d000096a0 con 0x7f2d0c0c9900 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.085+0000 7f2cfaffd640 1 -- 172.21.15.31:0/1630072763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00009800 con 0x7f2d0c0c9900 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.085+0000 7f2cfaffd640 1 -- 172.21.15.31:0/1630072763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00009b90 con 0x7f2d0c0c9900 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.085+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1f13a0 con 0x7f2d0c0c9900 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.085+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1f0ac0 con 0x7f2d0c0c9900 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.086+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c1f1b60 con 0x7f2d0c0c9900 2024-01-19T04:57:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.086+0000 7f2cfaffd640 1 -- 172.21.15.31:0/1630072763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2d00009cf0 con 0x7f2d0c0c9900 2024-01-19T04:57:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.086+0000 7f2cfaffd640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2cec06aed0 0x7f2cec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.086+0000 7f2cfaffd640 1 -- 172.21.15.31:0/1630072763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d000899d0 con 0x7f2d0c0c9900 2024-01-19T04:57:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.087+0000 7f2d0a59c640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2cec06aed0 0x7f2cec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.087+0000 7f2d0a59c640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2cec06aed0 0x7f2cec06d390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0079a0 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.091+0000 7f2cfaffd640 1 -- 172.21.15.31:0/1630072763 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2d00054950 con 0x7f2d0c0c9900 2024-01-19T04:57:27.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.265+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d0c074900 con 0x7f2d0c0c9900 2024-01-19T04:57:27.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.266+0000 7f2cfaffd640 1 -- 172.21.15.31:0/1630072763 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d0000b440 con 0x7f2d0c0c9900 2024-01-19T04:57:27.265 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2cec06aed0 msgr2=0x7f2cec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2cec06aed0 0x7f2cec06d390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0079a0 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0c9900 msgr2=0x7f2d0c0c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0c9900 0x7f2d0c0c8780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000d800 tx=0x7f2d0000dcc0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 shutdown_connections 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2cec06aed0 0x7f2cec06d390 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c15b4b0 0x7f2d0c0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0c0c9900 0x7f2d0c0c8780 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 --2- 172.21.15.31:0/1630072763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2d0c0720a0 0x7f2d0c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 >> 172.21.15.31:0/1630072763 conn(0x7f2d0c0b0da0 msgr2=0x7f2d0c0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 shutdown_connections 2024-01-19T04:57:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.268+0000 7f2d109c9640 1 -- 172.21.15.31:0/1630072763 wait complete. 2024-01-19T04:57:27.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:57:27.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.382+0000 7efd60410640 1 Processor -- start 2024-01-19T04:57:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.382+0000 7efd60410640 1 -- start start 2024-01-19T04:57:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd60410640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efd5815b6b0 0x7efd580d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd60410640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd580cce50 con 0x7efd580cd2e0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd60410640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd580cc240 con 0x7efd580ce090 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd60410640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd580d03b0 con 0x7efd5815b6b0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd5d9ab640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efd580cd2e0 0x7efd580cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48396/0 (socket says 172.21.15.31:48396) 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd5d9ab640 1 -- 172.21.15.31:0/1380853298 learned_addr learned my addr 172.21.15.31:0/1380853298 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd5e1ac640 1 --2- 172.21.15.31:0/1380853298 >> v2:172.21.15.137:3300/0 conn(0x7efd5815b6b0 0x7efd580d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521701290 0 0) 0x7efd580cce50 con 0x7efd580cd2e0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.383+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd40003520 con 0x7efd580cd2e0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2682453774 0 0) 0x7efd40003520 con 0x7efd580cd2e0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 >> v2:172.21.15.137:3300/0 conn(0x7efd5815b6b0 msgr2=0x7efd580d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 --2- 172.21.15.31:0/1380853298 >> v2:172.21.15.137:3300/0 conn(0x7efd5815b6b0 0x7efd580d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 >> v1:172.21.15.137:6789/0 conn(0x7efd580ce090 legacy=0x7efd580b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd580cc520 con 0x7efd580cd2e0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7efd4c001670 con 0x7efd580cd2e0 2024-01-19T04:57:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5e1ac640 1 --2- 172.21.15.31:0/1380853298 >> v2:172.21.15.137:3300/0 conn(0x7efd5815b6b0 0x7efd580d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 >> v1:172.21.15.31:6789/0 conn(0x7efd580cd2e0 legacy=0x7efd580cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd400039c0 0x7efd40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd40005180 0x7efd4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd580d03b0 con 0x7efd400047c0 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4000c060 con 0x7efd400039c0 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4000c220 con 0x7efd40005180 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5e1ac640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd400039c0 0x7efd40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5e9ad640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd40005180 0x7efd4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d9ab640 1 -- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd40005180 msgr2=0x7efd4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd40005180 0x7efd4000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d9ab640 1 -- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd400039c0 msgr2=0x7efd40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd400039c0 0x7efd40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5e9ad640 1 -- 172.21.15.31:0/1380853298 reap_dead start 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.384+0000 7efd5d9ab640 1 -- 172.21.15.31:0/1380853298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd40003520 con 0x7efd400047c0 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.385+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd40004c40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00c0e0 tx=0x7efd400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa38d40a491e46d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.385+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c005ad0 con 0x7efd400047c0 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.385+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c005c30 con 0x7efd400047c0 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.385+0000 7efd5d1aa640 1 -- 172.21.15.31:0/1380853298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02ca50 con 0x7efd400047c0 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.386+0000 7efd60410640 1 -- 172.21.15.31:0/1380853298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 msgr2=0x7efd40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.386+0000 7efd60410640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd40004c40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00c0e0 tx=0x7efd400057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.386+0000 7efd60410640 1 -- 172.21.15.31:0/1380853298 shutdown_connections 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.386+0000 7efd60410640 1 --2- 172.21.15.31:0/1380853298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd40004c40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.386+0000 7efd60410640 1 -- 172.21.15.31:0/1380853298 >> 172.21.15.31:0/1380853298 conn(0x7efd580d1100 msgr2=0x7efd580d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.386+0000 7efd60410640 1 -- 172.21.15.31:0/1380853298 shutdown_connections 2024-01-19T04:57:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.386+0000 7efd60410640 1 -- 172.21.15.31:0/1380853298 wait complete. 2024-01-19T04:57:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.387+0000 7efd60410640 1 Processor -- start 2024-01-19T04:57:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.387+0000 7efd60410640 1 -- start start 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd580b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5806d4c0 0x7efd580c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd580ca330 0x7efd5814d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5806d0a0 con 0x7efd400047c0 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5806c420 con 0x7efd5806d4c0 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5806c700 con 0x7efd580ca330 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd580b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd580b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59964/0 (socket says 172.21.15.31:59964) 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 -- 172.21.15.31:0/3448283853 learned_addr learned my addr 172.21.15.31:0/3448283853 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e9ad640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd580ca330 0x7efd5814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5806d4c0 0x7efd580c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 -- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd580ca330 msgr2=0x7efd5814d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd580ca330 0x7efd5814d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 -- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5806d4c0 msgr2=0x7efd580c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5806d4c0 0x7efd580c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 -- 172.21.15.31:0/3448283853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5806c9e0 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5806d4c0 0x7efd580c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd5e1ac640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd580b6100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efd48011fd0 tx=0x7efd4800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd56ffd640 1 -- 172.21.15.31:0/3448283853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800e040 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd56ffd640 1 -- 172.21.15.31:0/3448283853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4800f040 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5806ccc0 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd56ffd640 1 -- 172.21.15.31:0/3448283853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48003db0 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.388+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd580b6640 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.390+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd580b6890 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.390+0000 7efd56ffd640 1 -- 172.21.15.31:0/3448283853 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efd48004020 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.390+0000 7efd56ffd640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd4406aed0 0x7efd4406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.390+0000 7efd56ffd640 1 -- 172.21.15.31:0/3448283853 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd4808a130 con 0x7efd400047c0 2024-01-19T04:57:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.390+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd4406aed0 0x7efd4406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.391+0000 7efd5d9ab640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd4406aed0 0x7efd4406d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd4c034000 tx=0x7efd4c009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.395+0000 7efd56ffd640 1 -- 172.21.15.31:0/3448283853 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efd48055140 con 0x7efd400047c0 2024-01-19T04:57:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.575+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd58074900 con 0x7efd400047c0 2024-01-19T04:57:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.575+0000 7efd56ffd640 1 -- 172.21.15.31:0/3448283853 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd48058760 con 0x7efd400047c0 2024-01-19T04:57:27.575 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:57:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd4406aed0 msgr2=0x7efd4406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd4406aed0 0x7efd4406d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd4c034000 tx=0x7efd4c009030 comp rx=0 tx=0).stop 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 msgr2=0x7efd580b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd580b6100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efd48011fd0 tx=0x7efd4800f580 comp rx=0 tx=0).stop 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 shutdown_connections 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efd4406aed0 0x7efd4406d390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efd580ca330 0x7efd5814d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5806d4c0 0x7efd580c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 --2- 172.21.15.31:0/3448283853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd400047c0 0x7efd580b6100 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 >> 172.21.15.31:0/3448283853 conn(0x7efd580d1100 msgr2=0x7efd5815d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 shutdown_connections 2024-01-19T04:57:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.577+0000 7efd60410640 1 -- 172.21.15.31:0/3448283853 wait complete. 2024-01-19T04:57:27.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:57:27.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.691+0000 7f2893ca2640 1 Processor -- start 2024-01-19T04:57:27.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.692+0000 7f2893ca2640 1 -- start start 2024-01-19T04:57:27.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.692+0000 7f2893ca2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f288c0cc660 0x7f288c06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.692+0000 7f2893ca2640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c0c6130 con 0x7f288c15d340 2024-01-19T04:57:27.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.692+0000 7f2893ca2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c0ce630 con 0x7f288c06aaf0 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.692+0000 7f2893ca2640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288c06af80 con 0x7f288c0cc660 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2891a3e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f288c15d340 0x7f288c0cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48404/0 (socket says 172.21.15.31:48404) 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2891a3e640 1 -- 172.21.15.31:0/3238198361 learned_addr learned my addr 172.21.15.31:0/3238198361 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520303221 0 0) 0x7f288c0c6130 con 0x7f288c15d340 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2878003520 con 0x7f288c15d340 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f289123d640 1 --2- 172.21.15.31:0/3238198361 >> v2:172.21.15.137:3300/0 conn(0x7f288c0cc660 0x7f288c06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855864644 0 0) 0x7f2878003520 con 0x7f288c15d340 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 >> v2:172.21.15.137:3300/0 conn(0x7f288c0cc660 msgr2=0x7f288c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 --2- 172.21.15.31:0/3238198361 >> v2:172.21.15.137:3300/0 conn(0x7f288c0cc660 0x7f288c06a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 >> v1:172.21.15.137:6789/0 conn(0x7f288c06aaf0 legacy=0x7f288c071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0c5520 con 0x7f288c15d340 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f2888001670 con 0x7f288c15d340 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 >> v1:172.21.15.31:6789/0 conn(0x7f288c15d340 legacy=0x7f288c0cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f2878003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28780047c0 0x7f2878004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28780052a0 0x7f287800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c06af80 con 0x7f2878003970 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287800c170 con 0x7f28780052a0 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287800c330 con 0x7f28780047c0 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f2891a3e640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f2878003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f289223f640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28780052a0 0x7f287800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.693+0000 7f289123d640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28780047c0 0x7f2878004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2891a3e640 1 -- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28780047c0 msgr2=0x7f2878004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2891a3e640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28780047c0 0x7f2878004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2891a3e640 1 -- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28780052a0 msgr2=0x7f287800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2891a3e640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28780052a0 0x7f287800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2891a3e640 1 -- 172.21.15.31:0/3238198361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2878003520 con 0x7f2878003970 2024-01-19T04:57:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f289223f640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28780052a0 0x7f287800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f289123d640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28780047c0 0x7f2878004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f289223f640 1 -- 172.21.15.31:0/3238198361 reap_dead start 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2891a3e640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f2878003d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f288800c210 tx=0x7f28780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ec793f8068f449 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888005ad0 con 0x7f2878003970 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2888005c30 con 0x7f2878003970 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.694+0000 7f2890a3c640 1 -- 172.21.15.31:0/3238198361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288802cad0 con 0x7f2878003970 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.695+0000 7f2893ca2640 1 -- 172.21.15.31:0/3238198361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 msgr2=0x7f2878003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.695+0000 7f2893ca2640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f2878003d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f288800c210 tx=0x7f28780057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.695+0000 7f2893ca2640 1 -- 172.21.15.31:0/3238198361 shutdown_connections 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.695+0000 7f2893ca2640 1 --2- 172.21.15.31:0/3238198361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f2878003d90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.695+0000 7f2893ca2640 1 -- 172.21.15.31:0/3238198361 >> 172.21.15.31:0/3238198361 conn(0x7f288c0b1c50 msgr2=0x7f288c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.695+0000 7f2893ca2640 1 -- 172.21.15.31:0/3238198361 shutdown_connections 2024-01-19T04:57:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.695+0000 7f2893ca2640 1 -- 172.21.15.31:0/3238198361 wait complete. 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.696+0000 7f2893ca2640 1 Processor -- start 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.696+0000 7f2893ca2640 1 -- start start 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2893ca2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f288c158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2893ca2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f288c0ae450 0x7f288c15b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2893ca2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f288c0ae8b0 0x7f288c0c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2893ca2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c158c80 con 0x7f2878003970 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2893ca2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c15bee0 con 0x7f288c0ae8b0 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2893ca2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0ad690 con 0x7f288c0ae450 2024-01-19T04:57:27.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f288c158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f288c158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59980/0 (socket says 172.21.15.31:59980) 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 -- 172.21.15.31:0/1687880987 learned_addr learned my addr 172.21.15.31:0/1687880987 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f289123d640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f288c0ae450 0x7f288c15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f289223f640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f288c0ae8b0 0x7f288c0c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 -- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f288c0ae450 msgr2=0x7f288c15b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f288c0ae450 0x7f288c15b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 -- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f288c0ae8b0 msgr2=0x7f288c0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f288c0ae8b0 0x7f288c0c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f2891a3e640 1 -- 172.21.15.31:0/1687880987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0ad970 con 0x7f2878003970 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.697+0000 7f289123d640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f288c0ae450 0x7f288c15b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.698+0000 7f2891a3e640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f288c158740 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2888000f60 tx=0x7f288802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.698+0000 7f28827fc640 1 -- 172.21.15.31:0/1687880987 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288803c040 con 0x7f2878003970 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.698+0000 7f28827fc640 1 -- 172.21.15.31:0/1687880987 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288802cca0 con 0x7f2878003970 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.698+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c0adc50 con 0x7f2878003970 2024-01-19T04:57:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.698+0000 7f28827fc640 1 -- 172.21.15.31:0/1687880987 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888007270 con 0x7f2878003970 2024-01-19T04:57:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.698+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c1f3910 con 0x7f2878003970 2024-01-19T04:57:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.699+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c1f3aa0 con 0x7f2878003970 2024-01-19T04:57:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.699+0000 7f28827fc640 1 -- 172.21.15.31:0/1687880987 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2888032070 con 0x7f2878003970 2024-01-19T04:57:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.700+0000 7f28827fc640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f287006af00 0x7f287006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.700+0000 7f28827fc640 1 -- 172.21.15.31:0/1687880987 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28880aaa00 con 0x7f2878003970 2024-01-19T04:57:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.700+0000 7f289123d640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f287006af00 0x7f287006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.701+0000 7f289123d640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f287006af00 0x7f287006d3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f288c0b5740 tx=0x7f287c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:27.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.704+0000 7f28827fc640 1 -- 172.21.15.31:0/1687880987 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f28880758d0 con 0x7f2878003970 2024-01-19T04:57:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.886+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f288c074900 con 0x7f2878003970 2024-01-19T04:57:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.886+0000 7f28827fc640 1 -- 172.21.15.31:0/1687880987 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2888078ef0 con 0x7f2878003970 2024-01-19T04:57:27.886 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:57:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f287006af00 msgr2=0x7f287006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f287006af00 0x7f287006d3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f288c0b5740 tx=0x7f287c008040 comp rx=0 tx=0).stop 2024-01-19T04:57:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 msgr2=0x7f288c158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f288c158740 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2888000f60 tx=0x7f288802e070 comp rx=0 tx=0).stop 2024-01-19T04:57:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 shutdown_connections 2024-01-19T04:57:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f287006af00 0x7f287006d3c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f288c0ae8b0 0x7f288c0c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f288c0ae450 0x7f288c15b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 --2- 172.21.15.31:0/1687880987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2878003970 0x7f288c158740 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 >> 172.21.15.31:0/1687880987 conn(0x7f288c0b1c50 msgr2=0x7f288c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 shutdown_connections 2024-01-19T04:57:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:27.888+0000 7f2893ca2640 1 -- 172.21.15.31:0/1687880987 wait complete. 2024-01-19T04:57:27.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:57:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.001+0000 7f45b4e4f640 1 Processor -- start 2024-01-19T04:57:28.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.001+0000 7f45b4e4f640 1 -- start start 2024-01-19T04:57:28.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45b4e4f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f45b015d3a0 0x7f45b006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45b4e4f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45b00cf570 0x7f45b0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45b4e4f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b00cf9f0 con 0x7f45b015d780 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45b4e4f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b00cfcd0 con 0x7f45b00cf570 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45b4e4f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b00cffb0 con 0x7f45b015d3a0 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45a7fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45b00cf570 0x7f45b0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45aed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f45b015d780 0x7f45b00cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48416/0 (socket says 172.21.15.31:48416) 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45aed9d640 1 -- 172.21.15.31:0/1429144183 learned_addr learned my addr 172.21.15.31:0/1429144183 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45ae59c640 1 --2- 172.21.15.31:0/1429144183 >> v2:172.21.15.137:3300/0 conn(0x7f45b015d3a0 0x7f45b006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45a7fff640 1 -- 172.21.15.31:0/1429144183 >> v2:172.21.15.137:3300/0 conn(0x7f45b015d3a0 msgr2=0x7f45b006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45a7fff640 1 --2- 172.21.15.31:0/1429144183 >> v2:172.21.15.137:3300/0 conn(0x7f45b015d3a0 0x7f45b006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45a7fff640 1 -- 172.21.15.31:0/1429144183 >> v1:172.21.15.31:6789/0 conn(0x7f45b015d780 legacy=0x7f45b00cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45add9b640 1 -- 172.21.15.31:0/1429144183 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782673249 0 0) 0x7f45b00cf9f0 con 0x7f45b015d780 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.002+0000 7f45a7fff640 1 -- 172.21.15.31:0/1429144183 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00d0290 con 0x7f45b00cf570 2024-01-19T04:57:28.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.003+0000 7f45a7fff640 1 --2- 172.21.15.31:0/1429144183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00cf570 0x7f45b0161150 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f45b006dd90 tx=0x7f459802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b55a00537e13d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.003+0000 7f45add9b640 1 -- 172.21.15.31:0/1429144183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598031830 con 0x7f45b00cf570 2024-01-19T04:57:28.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.003+0000 7f45add9b640 1 -- 172.21.15.31:0/1429144183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4598031990 con 0x7f45b00cf570 2024-01-19T04:57:28.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.003+0000 7f45add9b640 1 -- 172.21.15.31:0/1429144183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598031d20 con 0x7f45b00cf570 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 -- 172.21.15.31:0/1429144183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00cf570 msgr2=0x7f45b0161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/1429144183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00cf570 0x7f45b0161150 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f45b006dd90 tx=0x7f459802b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 -- 172.21.15.31:0/1429144183 shutdown_connections 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/1429144183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00cf570 0x7f45b0161150 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/1429144183 >> v2:172.21.15.137:3300/0 conn(0x7f45b015d3a0 0x7f45b006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 -- 172.21.15.31:0/1429144183 >> 172.21.15.31:0/1429144183 conn(0x7f45b00d1100 msgr2=0x7f45b00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 -- 172.21.15.31:0/1429144183 shutdown_connections 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.004+0000 7f45b4e4f640 1 -- 172.21.15.31:0/1429144183 wait complete. 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.005+0000 7f45b4e4f640 1 Processor -- start 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.005+0000 7f45b4e4f640 1 -- start start 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45b4e4f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f45b00cf570 0x7f45b00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45b4e4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b015d3a0 0x7f45b00b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45b4e4f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b015d780 0x7f45b01f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45b4e4f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b00b1bf0 con 0x7f45b015d3a0 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45b4e4f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b00b1e70 con 0x7f45b015d780 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45b4e4f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b00b2150 con 0x7f45b00cf570 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b015d3a0 0x7f45b00b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b015d3a0 0x7f45b00b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60006/0 (socket says 172.21.15.31:60006) 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 -- 172.21.15.31:0/375910872 learned_addr learned my addr 172.21.15.31:0/375910872 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 -- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f45b00cf570 msgr2=0x7f45b00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45ae59c640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f45b00cf570 0x7f45b00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f45b00cf570 0x7f45b00b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 -- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b015d780 msgr2=0x7f45b01f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45aed9d640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b015d780 0x7f45b01f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b015d780 0x7f45b01f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 -- 172.21.15.31:0/375910872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598009400 con 0x7f45b015d3a0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.006+0000 7f45a7fff640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b015d3a0 0x7f45b00b16b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4598009890 tx=0x7f45980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.007+0000 7f45a6ffd640 1 -- 172.21.15.31:0/375910872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459803ba70 con 0x7f45b015d3a0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.007+0000 7f45a6ffd640 1 -- 172.21.15.31:0/375910872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459803bbf0 con 0x7f45b015d3a0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.007+0000 7f45a6ffd640 1 -- 172.21.15.31:0/375910872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598045400 con 0x7f45b015d3a0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.007+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b00b2430 con 0x7f45b015d3a0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.007+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b00b29f0 con 0x7f45b015d3a0 2024-01-19T04:57:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.008+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b0074900 con 0x7f45b015d3a0 2024-01-19T04:57:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.008+0000 7f45a6ffd640 1 -- 172.21.15.31:0/375910872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f459804e440 con 0x7f45b015d3a0 2024-01-19T04:57:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.008+0000 7f45a6ffd640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f459006aed0 0x7f459006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.008+0000 7f45a6ffd640 1 -- 172.21.15.31:0/375910872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45980ab800 con 0x7f45b015d3a0 2024-01-19T04:57:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.009+0000 7f45ae59c640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f459006aed0 0x7f459006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.009+0000 7f45ae59c640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f459006aed0 0x7f459006d390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009770 tx=0x7f45a0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.013+0000 7f45a6ffd640 1 -- 172.21.15.31:0/375910872 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4598076760 con 0x7f45b015d3a0 2024-01-19T04:57:28.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.204+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45b0067030 con 0x7f45b015d3a0 2024-01-19T04:57:28.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.204+0000 7f45a6ffd640 1 -- 172.21.15.31:0/375910872 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4598035080 con 0x7f45b015d3a0 2024-01-19T04:57:28.203 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f459006aed0 msgr2=0x7f459006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f459006aed0 0x7f459006d390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009770 tx=0x7f45a0008040 comp rx=0 tx=0).stop 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b015d3a0 msgr2=0x7f45b00b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b015d3a0 0x7f45b00b16b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4598009890 tx=0x7f45980046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 shutdown_connections 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f459006aed0 0x7f459006d390 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b015d780 0x7f45b01f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b015d3a0 0x7f45b00b16b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 --2- 172.21.15.31:0/375910872 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f45b00cf570 0x7f45b00b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 >> 172.21.15.31:0/375910872 conn(0x7f45b00d1100 msgr2=0x7f45b015eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 shutdown_connections 2024-01-19T04:57:28.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.206+0000 7f45b4e4f640 1 -- 172.21.15.31:0/375910872 wait complete. 2024-01-19T04:57:28.216 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:28.216 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:28.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:28.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.374+0000 7f324fd9e640 1 Processor -- start 2024-01-19T04:57:28.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.374+0000 7f324fd9e640 1 -- start start 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324fd9e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32480cfd10 0x7f32480d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324fd9e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480d06b0 con 0x7f32480d0ef0 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324fd9e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480d1b90 con 0x7f3248160bb0 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324fd9e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32480d1e70 con 0x7f32480cfd10 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324d339640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32480cfd10 0x7f32480d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324e33b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f32480d0ef0 0x7f32480d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48422/0 (socket says 172.21.15.31:48422) 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324d339640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32480cfd10 0x7f32480d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60012/0 (socket says 172.21.15.31:60012) 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324e33b640 1 -- 172.21.15.31:0/222808488 learned_addr learned my addr 172.21.15.31:0/222808488 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324d339640 1 -- 172.21.15.31:0/222808488 >> v1:172.21.15.137:6789/0 conn(0x7f3248160bb0 legacy=0x7f32480b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324d339640 1 -- 172.21.15.31:0/222808488 >> v1:172.21.15.31:6789/0 conn(0x7f32480d0ef0 legacy=0x7f32480d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324d339640 1 -- 172.21.15.31:0/222808488 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480d2150 con 0x7f32480cfd10 2024-01-19T04:57:28.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.375+0000 7f324cb38640 1 -- 172.21.15.31:0/222808488 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057656120 0 0) 0x7f32480d06b0 con 0x7f32480d0ef0 2024-01-19T04:57:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.376+0000 7f324d339640 1 --2- 172.21.15.31:0/222808488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32480cfd10 0x7f32480d0170 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f32380097a0 tx=0x7f323802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0e20a8e0cfc18f server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.376+0000 7f324cb38640 1 -- 172.21.15.31:0/222808488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238030610 con 0x7f32480cfd10 2024-01-19T04:57:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.376+0000 7f324cb38640 1 -- 172.21.15.31:0/222808488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238030770 con 0x7f32480cfd10 2024-01-19T04:57:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.376+0000 7f324cb38640 1 -- 172.21.15.31:0/222808488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238030b00 con 0x7f32480cfd10 2024-01-19T04:57:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.377+0000 7f324fd9e640 1 -- 172.21.15.31:0/222808488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32480cfd10 msgr2=0x7f32480d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.377+0000 7f324fd9e640 1 --2- 172.21.15.31:0/222808488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32480cfd10 0x7f32480d0170 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f32380097a0 tx=0x7f323802b960 comp rx=0 tx=0).stop 2024-01-19T04:57:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.377+0000 7f324fd9e640 1 -- 172.21.15.31:0/222808488 shutdown_connections 2024-01-19T04:57:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.377+0000 7f324fd9e640 1 --2- 172.21.15.31:0/222808488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32480cfd10 0x7f32480d0170 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.377+0000 7f324fd9e640 1 -- 172.21.15.31:0/222808488 >> 172.21.15.31:0/222808488 conn(0x7f324809c600 msgr2=0x7f32480d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.377+0000 7f324fd9e640 1 -- 172.21.15.31:0/222808488 shutdown_connections 2024-01-19T04:57:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.377+0000 7f324fd9e640 1 -- 172.21.15.31:0/222808488 wait complete. 2024-01-19T04:57:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.378+0000 7f324fd9e640 1 Processor -- start 2024-01-19T04:57:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.378+0000 7f324fd9e640 1 -- start start 2024-01-19T04:57:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324fd9e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f32480cfd10 0x7f32480b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324fd9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32480d0ef0 0x7f324814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324fd9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248160bb0 0x7f3248159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324fd9e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480b62f0 con 0x7f3248160bb0 2024-01-19T04:57:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324fd9e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248157910 con 0x7f32480d0ef0 2024-01-19T04:57:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324fd9e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32480b7e50 con 0x7f32480cfd10 2024-01-19T04:57:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248160bb0 0x7f3248159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248160bb0 0x7f3248159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60028/0 (socket says 172.21.15.31:60028) 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 -- 172.21.15.31:0/3880449208 learned_addr learned my addr 172.21.15.31:0/3880449208 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324d339640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32480d0ef0 0x7f324814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324db3a640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f32480cfd10 0x7f32480b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 -- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f32480cfd10 msgr2=0x7f32480b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f32480cfd10 0x7f32480b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 -- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32480d0ef0 msgr2=0x7f324814e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32480d0ef0 0x7f324814e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 -- 172.21.15.31:0/3880449208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238009400 con 0x7f3248160bb0 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324db3a640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f32480cfd10 0x7f32480b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324e33b640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248160bb0 0x7f3248159f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f324000c730 tx=0x7f324000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f323e7fc640 1 -- 172.21.15.31:0/3880449208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324000ce40 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f323e7fc640 1 -- 172.21.15.31:0/3880449208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324000a640 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32480b6710 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.379+0000 7f323e7fc640 1 -- 172.21.15.31:0/3880449208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324000aa30 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.380+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32480b5c30 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.380+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3248067b50 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.381+0000 7f323e7fc640 1 -- 172.21.15.31:0/3880449208 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f324000ab90 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.381+0000 7f323e7fc640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f322c06af00 0x7f322c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.382+0000 7f323e7fc640 1 -- 172.21.15.31:0/3880449208 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3240088c40 con 0x7f3248160bb0 2024-01-19T04:57:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.382+0000 7f324db3a640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f322c06af00 0x7f322c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.382+0000 7f324db3a640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f322c06af00 0x7f322c06d3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3244007fd0 tx=0x7f3244009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.386+0000 7f323e7fc640 1 -- 172.21.15.31:0/3880449208 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3240053c50 con 0x7f3248160bb0 2024-01-19T04:57:28.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.565+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32480b6430 con 0x7f3248160bb0 2024-01-19T04:57:28.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.566+0000 7f323e7fc640 1 -- 172.21.15.31:0/3880449208 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f32480b6430 con 0x7f3248160bb0 2024-01-19T04:57:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:28.566 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f322c06af00 msgr2=0x7f322c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f322c06af00 0x7f322c06d3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3244007fd0 tx=0x7f3244009040 comp rx=0 tx=0).stop 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248160bb0 msgr2=0x7f3248159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248160bb0 0x7f3248159f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f324000c730 tx=0x7f324000cbf0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 shutdown_connections 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f322c06af00 0x7f322c06d3c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248160bb0 0x7f3248159f20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32480d0ef0 0x7f324814e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 --2- 172.21.15.31:0/3880449208 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f32480cfd10 0x7f32480b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 >> 172.21.15.31:0/3880449208 conn(0x7f324809c600 msgr2=0x7f324815e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 shutdown_connections 2024-01-19T04:57:28.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.568+0000 7f324fd9e640 1 -- 172.21.15.31:0/3880449208 wait complete. 2024-01-19T04:57:28.578 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:57:10.830975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:57:11.850371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:28.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa731a5c640 1 Processor -- start 2024-01-19T04:57:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa731a5c640 1 -- start start 2024-01-19T04:57:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa731a5c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa72c0cb2e0 0x7fa72c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa731a5c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa72c06d6b0 0x7fa72c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa731a5c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c15ece0 con 0x7fa72c0716f0 2024-01-19T04:57:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa731a5c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c06e060 con 0x7fa72c06d6b0 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa731a5c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c071c20 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.684+0000 7fa72b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa72c06d6b0 0x7fa72c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa72a7fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa72c0716f0 0x7fa72c15afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48438/0 (socket says 172.21.15.31:48438) 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa72a7fc640 1 -- 172.21.15.31:0/2603950062 learned_addr learned my addr 172.21.15.31:0/2603950062 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa72b7fe640 1 -- 172.21.15.31:0/2603950062 >> v2:172.21.15.137:3300/0 conn(0x7fa72c0cb2e0 msgr2=0x7fa72c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa72affd640 1 --2- 172.21.15.31:0/2603950062 >> v2:172.21.15.137:3300/0 conn(0x7fa72c0cb2e0 0x7fa72c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa72b7fe640 1 --2- 172.21.15.31:0/2603950062 >> v2:172.21.15.137:3300/0 conn(0x7fa72c0cb2e0 0x7fa72c15e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa729ffb640 1 -- 172.21.15.31:0/2603950062 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51853455 0 0) 0x7fa72c15ece0 con 0x7fa72c0716f0 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa72b7fe640 1 -- 172.21.15.31:0/2603950062 >> v1:172.21.15.31:6789/0 conn(0x7fa72c0716f0 legacy=0x7fa72c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.685+0000 7fa72b7fe640 1 -- 172.21.15.31:0/2603950062 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c071f00 con 0x7fa72c06d6b0 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa72b7fe640 1 --2- 172.21.15.31:0/2603950062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c06d6b0 0x7fa72c06db20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0cff60 tx=0x7fa72402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f583eca27a3d5c72 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa729ffb640 1 -- 172.21.15.31:0/2603950062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724031830 con 0x7fa72c06d6b0 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa729ffb640 1 -- 172.21.15.31:0/2603950062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724031990 con 0x7fa72c06d6b0 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa729ffb640 1 -- 172.21.15.31:0/2603950062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724031d20 con 0x7fa72c06d6b0 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 -- 172.21.15.31:0/2603950062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c06d6b0 msgr2=0x7fa72c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2603950062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c06d6b0 0x7fa72c06db20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0cff60 tx=0x7fa72402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 -- 172.21.15.31:0/2603950062 shutdown_connections 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2603950062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c06d6b0 0x7fa72c06db20 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2603950062 >> v2:172.21.15.137:3300/0 conn(0x7fa72c0cb2e0 0x7fa72c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 -- 172.21.15.31:0/2603950062 >> 172.21.15.31:0/2603950062 conn(0x7fa72c06aa00 msgr2=0x7fa72c06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 -- 172.21.15.31:0/2603950062 shutdown_connections 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.686+0000 7fa731a5c640 1 -- 172.21.15.31:0/2603950062 wait complete. 2024-01-19T04:57:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.687+0000 7fa731a5c640 1 Processor -- start 2024-01-19T04:57:28.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa731a5c640 1 -- start start 2024-01-19T04:57:28.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa731a5c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa72c06d6b0 0x7fa72c157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa731a5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72c0716f0 0x7fa72c157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa731a5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c0cb2e0 0x7fa72c158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa731a5c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c158550 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa731a5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c159d60 con 0x7fa72c0716f0 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa731a5c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c1564f0 con 0x7fa72c06d6b0 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa72b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c0cb2e0 0x7fa72c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa72b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c0cb2e0 0x7fa72c158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60042/0 (socket says 172.21.15.31:60042) 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.688+0000 7fa72b7fe640 1 -- 172.21.15.31:0/2241231655 learned_addr learned my addr 172.21.15.31:0/2241231655 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72a7fc640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72c0716f0 0x7fa72c157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72affd640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa72c06d6b0 0x7fa72c157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72b7fe640 1 -- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa72c06d6b0 msgr2=0x7fa72c157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72b7fe640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa72c06d6b0 0x7fa72c157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72b7fe640 1 -- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72c0716f0 msgr2=0x7fa72c157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72b7fe640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72c0716f0 0x7fa72c157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72b7fe640 1 -- 172.21.15.31:0/2241231655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724009400 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa72b7fe640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c0cb2e0 0x7fa72c158010 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa72402bee0 tx=0x7fa7240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa713fff640 1 -- 172.21.15.31:0/2241231655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72403b9f0 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa713fff640 1 -- 172.21.15.31:0/2241231655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72403bb70 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c1567d0 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa713fff640 1 -- 172.21.15.31:0/2241231655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724045400 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.689+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c156d90 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.690+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c074900 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.690+0000 7fa713fff640 1 -- 172.21.15.31:0/2241231655 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa72404e440 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.691+0000 7fa713fff640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa71806aed0 0x7fa71806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.691+0000 7fa713fff640 1 -- 172.21.15.31:0/2241231655 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7240ab890 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.691+0000 7fa72affd640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa71806aed0 0x7fa71806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.692+0000 7fa72affd640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa71806aed0 0x7fa71806d390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0070b0 tx=0x7fa71c007040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.695+0000 7fa713fff640 1 -- 172.21.15.31:0/2241231655 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa7240767f0 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.871+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa72c0b31d0 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.872+0000 7fa713fff640 1 -- 172.21.15.31:0/2241231655 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa724079e10 con 0x7fa72c0cb2e0 2024-01-19T04:57:28.871 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa71806aed0 msgr2=0x7fa71806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa71806aed0 0x7fa71806d390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0070b0 tx=0x7fa71c007040 comp rx=0 tx=0).stop 2024-01-19T04:57:28.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c0cb2e0 msgr2=0x7fa72c158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c0cb2e0 0x7fa72c158010 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa72402bee0 tx=0x7fa7240046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 shutdown_connections 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa71806aed0 0x7fa71806d390 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa72c0cb2e0 0x7fa72c158010 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72c0716f0 0x7fa72c157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 --2- 172.21.15.31:0/2241231655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa72c06d6b0 0x7fa72c157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 >> 172.21.15.31:0/2241231655 conn(0x7fa72c06aa00 msgr2=0x7fa72c153710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 shutdown_connections 2024-01-19T04:57:28.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.874+0000 7fa731a5c640 1 -- 172.21.15.31:0/2241231655 wait complete. 2024-01-19T04:57:28.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:57:28.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.988+0000 7f28b2f68640 1 Processor -- start 2024-01-19T04:57:28.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.988+0000 7f28b2f68640 1 -- start start 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b2f68640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d18e0 0x7f28ac15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b2f68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f28ac0d0ef0 0x7f28ac0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b2f68640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ac15dee0 con 0x7f28ac0d3e50 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b2f68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ac15e1c0 con 0x7f28ac0d0ef0 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b2f68640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac15e4a0 con 0x7f28ac0d18e0 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b1505640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f28ac0d0ef0 0x7f28ac0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28abfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f28ac0d3e50 0x7f28ac0d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48452/0 (socket says 172.21.15.31:48452) 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b1505640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f28ac0d0ef0 0x7f28ac0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60054/0 (socket says 172.21.15.31:60054) 2024-01-19T04:57:28.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28abfff640 1 -- 172.21.15.31:0/1318061273 learned_addr learned my addr 172.21.15.31:0/1318061273 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b0d04640 1 --2- 172.21.15.31:0/1318061273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d18e0 0x7f28ac15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b1505640 1 -- 172.21.15.31:0/1318061273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d18e0 msgr2=0x7f28ac15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b1505640 1 --2- 172.21.15.31:0/1318061273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d18e0 0x7f28ac15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b1505640 1 -- 172.21.15.31:0/1318061273 >> v1:172.21.15.31:6789/0 conn(0x7f28ac0d3e50 legacy=0x7f28ac0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28b1505640 1 -- 172.21.15.31:0/1318061273 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac15e780 con 0x7f28ac0d0ef0 2024-01-19T04:57:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.989+0000 7f28ab7fe640 1 -- 172.21.15.31:0/1318061273 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140162994 0 0) 0x7f28ac15dee0 con 0x7f28ac0d3e50 2024-01-19T04:57:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.990+0000 7f28b1505640 1 --2- 172.21.15.31:0/1318061273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d0ef0 0x7f28ac0cdfe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009d90 tx=0x7f28a402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e588e09365d220 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.990+0000 7f28ab7fe640 1 -- 172.21.15.31:0/1318061273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a402cc50 con 0x7f28ac0d0ef0 2024-01-19T04:57:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.990+0000 7f28ab7fe640 1 -- 172.21.15.31:0/1318061273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a402cdb0 con 0x7f28ac0d0ef0 2024-01-19T04:57:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.990+0000 7f28ab7fe640 1 -- 172.21.15.31:0/1318061273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4034660 con 0x7f28ac0d0ef0 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 -- 172.21.15.31:0/1318061273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d0ef0 msgr2=0x7f28ac0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1318061273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d0ef0 0x7f28ac0cdfe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009d90 tx=0x7f28a402bea0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 -- 172.21.15.31:0/1318061273 shutdown_connections 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1318061273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d0ef0 0x7f28ac0cdfe0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1318061273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d18e0 0x7f28ac15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 -- 172.21.15.31:0/1318061273 >> 172.21.15.31:0/1318061273 conn(0x7f28ac09c600 msgr2=0x7f28ac0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 -- 172.21.15.31:0/1318061273 shutdown_connections 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.991+0000 7f28b2f68640 1 -- 172.21.15.31:0/1318061273 wait complete. 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.992+0000 7f28b2f68640 1 Processor -- start 2024-01-19T04:57:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.992+0000 7f28b2f68640 1 -- start start 2024-01-19T04:57:28.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b2f68640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d0ef0 0x7f28ac0b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b2f68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d18e0 0x7f28ac0c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b2f68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ac0d3e50 0x7f28ac151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b2f68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c7f90 con 0x7f28ac0d18e0 2024-01-19T04:57:28.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b2f68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c7380 con 0x7f28ac0d3e50 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b2f68640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c7660 con 0x7f28ac0d0ef0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d18e0 0x7f28ac0c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d18e0 0x7f28ac0c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60062/0 (socket says 172.21.15.31:60062) 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 -- 172.21.15.31:0/1125025502 learned_addr learned my addr 172.21.15.31:0/1125025502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b0d04640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d0ef0 0x7f28ac0b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b1505640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ac0d3e50 0x7f28ac151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 -- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d0ef0 msgr2=0x7f28ac0b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d0ef0 0x7f28ac0b8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 -- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ac0d3e50 msgr2=0x7f28ac151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ac0d3e50 0x7f28ac151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 -- 172.21.15.31:0/1125025502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a4009a70 con 0x7f28ac0d18e0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28b1505640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ac0d3e50 0x7f28ac151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.993+0000 7f28abfff640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d18e0 0x7f28ac0c8420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f289800c430 tx=0x7f289800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.994+0000 7f28a97fa640 1 -- 172.21.15.31:0/1125025502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980074e0 con 0x7f28ac0d18e0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.994+0000 7f28a97fa640 1 -- 172.21.15.31:0/1125025502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2898007660 con 0x7f28ac0d18e0 2024-01-19T04:57:28.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.994+0000 7f28a97fa640 1 -- 172.21.15.31:0/1125025502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898004a20 con 0x7f28ac0d18e0 2024-01-19T04:57:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.994+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac0c7940 con 0x7f28ac0d18e0 2024-01-19T04:57:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.994+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac151b40 con 0x7f28ac0d18e0 2024-01-19T04:57:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.995+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac074900 con 0x7f28ac0d18e0 2024-01-19T04:57:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.995+0000 7f28a97fa640 1 -- 172.21.15.31:0/1125025502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2898004b80 con 0x7f28ac0d18e0 2024-01-19T04:57:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.995+0000 7f28a97fa640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f289006aed0 0x7f289006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:28.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.995+0000 7f28a97fa640 1 -- 172.21.15.31:0/1125025502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2898089410 con 0x7f28ac0d18e0 2024-01-19T04:57:28.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.996+0000 7f28b0d04640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f289006aed0 0x7f289006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:28.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:28.996+0000 7f28b0d04640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f289006aed0 0x7f289006d390 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f28a0006fd0 tx=0x7f28a0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:28.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.000+0000 7f28a97fa640 1 -- 172.21.15.31:0/1125025502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f28980542e0 con 0x7f28ac0d18e0 2024-01-19T04:57:29.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.176+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28ac0d09a0 con 0x7f28ac0d18e0 2024-01-19T04:57:29.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.177+0000 7f28a97fa640 1 -- 172.21.15.31:0/1125025502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2898057900 con 0x7f28ac0d18e0 2024-01-19T04:57:29.176 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f289006aed0 msgr2=0x7f289006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f289006aed0 0x7f289006d390 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f28a0006fd0 tx=0x7f28a0008040 comp rx=0 tx=0).stop 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d18e0 msgr2=0x7f28ac0c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d18e0 0x7f28ac0c8420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f289800c430 tx=0x7f289800c8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 shutdown_connections 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f289006aed0 0x7f289006d390 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ac0d3e50 0x7f28ac151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac0d18e0 0x7f28ac0c8420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 --2- 172.21.15.31:0/1125025502 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ac0d0ef0 0x7f28ac0b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 >> 172.21.15.31:0/1125025502 conn(0x7f28ac09c600 msgr2=0x7f28ac06c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 shutdown_connections 2024-01-19T04:57:29.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.179+0000 7f28b2f68640 1 -- 172.21.15.31:0/1125025502 wait complete. 2024-01-19T04:57:29.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:57:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.294+0000 7f64c4d25640 1 Processor -- start 2024-01-19T04:57:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.294+0000 7f64c4d25640 1 -- start start 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64c4d25640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f64c015b8c0 0x7f64c015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64c4d25640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c0072110 0x7f64c013ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64c4d25640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c015f280 con 0x7f64c013e8e0 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64c4d25640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c015fd50 con 0x7f64c015b8c0 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64c4d25640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c0071570 con 0x7f64c0072110 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64bed9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c0072110 0x7f64c013ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64be59c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f64c015b8c0 0x7f64c015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64b7fff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f64c013e8e0 0x7f64c0071a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.31:49110/0 (socket says 172.21.15.31:49110) 2024-01-19T04:57:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64be59c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f64c015b8c0 0x7f64c015ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:34192/0 (socket says 172.21.15.31:34192) 2024-01-19T04:57:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64b7fff640 1 -- 172.21.15.31:0/2844123599 learned_addr learned my addr 172.21.15.31:0/2844123599 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.295+0000 7f64bed9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c0072110 0x7f64c013ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:44990/0 (socket says 172.21.15.31:44990) 2024-01-19T04:57:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.296+0000 7f64bed9d640 1 -- 172.21.15.31:0/2844123599 >> v2:172.21.15.137:3300/0 conn(0x7f64c015b8c0 msgr2=0x7f64c015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.296+0000 7f64bed9d640 1 --2- 172.21.15.31:0/2844123599 >> v2:172.21.15.137:3300/0 conn(0x7f64c015b8c0 0x7f64c015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.296+0000 7f64bed9d640 1 -- 172.21.15.31:0/2844123599 >> v1:172.21.15.137:6789/0 conn(0x7f64c013e8e0 legacy=0x7f64c0071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.296+0000 7f64bed9d640 1 -- 172.21.15.31:0/2844123599 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c015f4b0 con 0x7f64c0072110 2024-01-19T04:57:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.296+0000 7f64bed9d640 1 --2- 172.21.15.31:0/2844123599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c0072110 0x7f64c013ed60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009d90 tx=0x7f64b802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc514c5c0995f50 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64bdd9b640 1 -- 172.21.15.31:0/2844123599 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b802cc50 con 0x7f64c0072110 2024-01-19T04:57:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64bdd9b640 1 -- 172.21.15.31:0/2844123599 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b802cdb0 con 0x7f64c0072110 2024-01-19T04:57:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64bdd9b640 1 -- 172.21.15.31:0/2844123599 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8034660 con 0x7f64c0072110 2024-01-19T04:57:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 -- 172.21.15.31:0/2844123599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c0072110 msgr2=0x7f64c013ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 --2- 172.21.15.31:0/2844123599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c0072110 0x7f64c013ed60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009d90 tx=0x7f64b802bea0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 -- 172.21.15.31:0/2844123599 shutdown_connections 2024-01-19T04:57:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 --2- 172.21.15.31:0/2844123599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c0072110 0x7f64c013ed60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 --2- 172.21.15.31:0/2844123599 >> v2:172.21.15.137:3300/0 conn(0x7f64c015b8c0 0x7f64c015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 -- 172.21.15.31:0/2844123599 >> 172.21.15.31:0/2844123599 conn(0x7f64c00b1c50 msgr2=0x7f64c00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 -- 172.21.15.31:0/2844123599 shutdown_connections 2024-01-19T04:57:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.297+0000 7f64c4d25640 1 -- 172.21.15.31:0/2844123599 wait complete. 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.298+0000 7f64c4d25640 1 Processor -- start 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64c4d25640 1 -- start start 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64c4d25640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0072110 0x7f64c01595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64c4d25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c013e8e0 0x7f64c00c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64c4d25640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c015b8c0 0x7f64c00c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64c4d25640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00c8670 con 0x7f64c0072110 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64c4d25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00c8950 con 0x7f64c013e8e0 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64c4d25640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00c8c30 con 0x7f64c015b8c0 2024-01-19T04:57:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64be59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0072110 0x7f64c01595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64be59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0072110 0x7f64c01595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60078/0 (socket says 172.21.15.31:60078) 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.299+0000 7f64be59c640 1 -- 172.21.15.31:0/1876217461 learned_addr learned my addr 172.21.15.31:0/1876217461 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64b7fff640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c013e8e0 0x7f64c00c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64bed9d640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c015b8c0 0x7f64c00c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64be59c640 1 -- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c015b8c0 msgr2=0x7f64c00c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64be59c640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c015b8c0 0x7f64c00c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64be59c640 1 -- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c013e8e0 msgr2=0x7f64c00c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64be59c640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c013e8e0 0x7f64c00c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64be59c640 1 -- 172.21.15.31:0/1876217461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009a70 con 0x7f64c0072110 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64be59c640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0072110 0x7f64c01595a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f64b000e9b0 tx=0x7f64b000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64b6ffd640 1 -- 172.21.15.31:0/1876217461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0009670 con 0x7f64c0072110 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64b6ffd640 1 -- 172.21.15.31:0/1876217461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b000c9a0 con 0x7f64c0072110 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c00c8f10 con 0x7f64c0072110 2024-01-19T04:57:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64b6ffd640 1 -- 172.21.15.31:0/1876217461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b000cd50 con 0x7f64c0072110 2024-01-19T04:57:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.300+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c0159ae0 con 0x7f64c0072110 2024-01-19T04:57:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.301+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c00cac00 con 0x7f64c0072110 2024-01-19T04:57:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.301+0000 7f64b6ffd640 1 -- 172.21.15.31:0/1876217461 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f64b0013470 con 0x7f64c0072110 2024-01-19T04:57:29.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.302+0000 7f64b6ffd640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f64a006aed0 0x7f64a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.302+0000 7f64b6ffd640 1 -- 172.21.15.31:0/1876217461 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64b0089890 con 0x7f64c0072110 2024-01-19T04:57:29.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.302+0000 7f64b7fff640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f64a006aed0 0x7f64a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.303+0000 7f64b7fff640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f64a006aed0 0x7f64a006d390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64c0158290 tx=0x7f64a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.306+0000 7f64b6ffd640 1 -- 172.21.15.31:0/1876217461 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f64b0054760 con 0x7f64c0072110 2024-01-19T04:57:29.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.483+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64c0074900 con 0x7f64c0072110 2024-01-19T04:57:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.483+0000 7f64b6ffd640 1 -- 172.21.15.31:0/1876217461 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64b00170a0 con 0x7f64c0072110 2024-01-19T04:57:29.483 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f64a006aed0 msgr2=0x7f64a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f64a006aed0 0x7f64a006d390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64c0158290 tx=0x7f64a8009040 comp rx=0 tx=0).stop 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0072110 msgr2=0x7f64c01595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0072110 0x7f64c01595a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f64b000e9b0 tx=0x7f64b000ee70 comp rx=0 tx=0).stop 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 shutdown_connections 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f64a006aed0 0x7f64a006d390 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f64c015b8c0 0x7f64c00c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c013e8e0 0x7f64c00c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 --2- 172.21.15.31:0/1876217461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0072110 0x7f64c01595a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 >> 172.21.15.31:0/1876217461 conn(0x7f64c00b1c50 msgr2=0x7f64c015c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 shutdown_connections 2024-01-19T04:57:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.485+0000 7f64c4d25640 1 -- 172.21.15.31:0/1876217461 wait complete. 2024-01-19T04:57:29.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:57:29.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.600+0000 7f242f207640 1 Processor -- start 2024-01-19T04:57:29.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.600+0000 7f242f207640 1 -- start start 2024-01-19T04:57:29.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242f207640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f242815d700 0x7f24280d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242f207640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280cef10 con 0x7f24280d1fe0 2024-01-19T04:57:29.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242f207640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280ce300 con 0x7f24280d11b0 2024-01-19T04:57:29.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242f207640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24280ce5e0 con 0x7f242815d700 2024-01-19T04:57:29.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242cfa3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f242815d700 0x7f24280d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242d7a4640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f24280d1fe0 0x7f24280b2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58928/0 (socket says 172.21.15.31:58928) 2024-01-19T04:57:29.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242d7a4640 1 -- 172.21.15.31:0/824068409 learned_addr learned my addr 172.21.15.31:0/824068409 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:29.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242cfa3640 1 -- 172.21.15.31:0/824068409 >> v1:172.21.15.137:6789/0 conn(0x7f24280d11b0 legacy=0x7f24280d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:29.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242cfa3640 1 -- 172.21.15.31:0/824068409 >> v1:172.21.15.31:6789/0 conn(0x7f24280d1fe0 legacy=0x7f24280b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.601+0000 7f242cfa3640 1 -- 172.21.15.31:0/824068409 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280ce8c0 con 0x7f242815d700 2024-01-19T04:57:29.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.602+0000 7f242cfa3640 1 --2- 172.21.15.31:0/824068409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242815d700 0x7f24280d3e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a0a244117977a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:29.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.602+0000 7f24277fe640 1 -- 172.21.15.31:0/824068409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418030610 con 0x7f242815d700 2024-01-19T04:57:29.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.602+0000 7f24277fe640 1 -- 172.21.15.31:0/824068409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418030770 con 0x7f242815d700 2024-01-19T04:57:29.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.602+0000 7f24277fe640 1 -- 172.21.15.31:0/824068409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418030b00 con 0x7f242815d700 2024-01-19T04:57:29.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.603+0000 7f242f207640 1 -- 172.21.15.31:0/824068409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242815d700 msgr2=0x7f24280d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.603+0000 7f242f207640 1 --2- 172.21.15.31:0/824068409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242815d700 0x7f24280d3e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802b960 comp rx=0 tx=0).stop 2024-01-19T04:57:29.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.603+0000 7f242f207640 1 -- 172.21.15.31:0/824068409 shutdown_connections 2024-01-19T04:57:29.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.603+0000 7f242f207640 1 --2- 172.21.15.31:0/824068409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242815d700 0x7f24280d3e10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.603+0000 7f242f207640 1 -- 172.21.15.31:0/824068409 >> 172.21.15.31:0/824068409 conn(0x7f24280d32f0 msgr2=0x7f24280d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:29.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.603+0000 7f242f207640 1 -- 172.21.15.31:0/824068409 shutdown_connections 2024-01-19T04:57:29.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.603+0000 7f242f207640 1 -- 172.21.15.31:0/824068409 wait complete. 2024-01-19T04:57:29.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.604+0000 7f242f207640 1 Processor -- start 2024-01-19T04:57:29.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.604+0000 7f242f207640 1 -- start start 2024-01-19T04:57:29.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242f207640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24280d11b0 0x7f242815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242f207640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24280d1fe0 0x7f242815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242f207640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242815d700 0x7f242815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242f207640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280b5e70 con 0x7f24280d1fe0 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242f207640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280b6110 con 0x7f242815d700 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242f207640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280b63f0 con 0x7f24280d11b0 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24280d1fe0 0x7f242815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24280d1fe0 0x7f242815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45658/0 (socket says 172.21.15.31:45658) 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 -- 172.21.15.31:0/4112789405 learned_addr learned my addr 172.21.15.31:0/4112789405 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242cfa3640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24280d11b0 0x7f242815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242d7a4640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242815d700 0x7f242815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 -- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24280d11b0 msgr2=0x7f242815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24280d11b0 0x7f242815a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 -- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242815d700 msgr2=0x7f242815bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242815d700 0x7f242815bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 -- 172.21.15.31:0/4112789405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418009400 con 0x7f24280d1fe0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242cfa3640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24280d11b0 0x7f242815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f242d7a4640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242815d700 0x7f242815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.605+0000 7f2427fff640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24280d1fe0 0x7f242815b8b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c410 tx=0x7f241c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.606+0000 7f24257fa640 1 -- 172.21.15.31:0/4112789405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0035f0 con 0x7f24280d1fe0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.606+0000 7f24257fa640 1 -- 172.21.15.31:0/4112789405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c003750 con 0x7f24280d1fe0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.606+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280b66d0 con 0x7f24280d1fe0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.606+0000 7f24257fa640 1 -- 172.21.15.31:0/4112789405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c003b40 con 0x7f24280d1fe0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.606+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242815a6d0 con 0x7f24280d1fe0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.607+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428074900 con 0x7f24280d1fe0 2024-01-19T04:57:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.607+0000 7f24257fa640 1 -- 172.21.15.31:0/4112789405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f241c003ca0 con 0x7f24280d1fe0 2024-01-19T04:57:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.608+0000 7f24257fa640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f241406af00 0x7f241406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.608+0000 7f24257fa640 1 -- 172.21.15.31:0/4112789405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f241c088cb0 con 0x7f24280d1fe0 2024-01-19T04:57:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.608+0000 7f242cfa3640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f241406af00 0x7f241406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.608+0000 7f242cfa3640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f241406af00 0x7f241406d3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2418036000 tx=0x7f241803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:29.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.612+0000 7f24257fa640 1 -- 172.21.15.31:0/4112789405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f241c053b80 con 0x7f24280d1fe0 2024-01-19T04:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.806+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f24280d16b0 con 0x7f24280d1fe0 2024-01-19T04:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.806+0000 7f24257fa640 1 -- 172.21.15.31:0/4112789405 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f241c017020 con 0x7f24280d1fe0 2024-01-19T04:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f241406af00 msgr2=0x7f241406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f241406af00 0x7f241406d3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2418036000 tx=0x7f241803b040 comp rx=0 tx=0).stop 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24280d1fe0 msgr2=0x7f242815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24280d1fe0 0x7f242815b8b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c410 tx=0x7f241c00c8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 shutdown_connections 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f241406af00 0x7f241406d3c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242815d700 0x7f242815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24280d1fe0 0x7f242815b8b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 --2- 172.21.15.31:0/4112789405 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24280d11b0 0x7f242815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:29.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 >> 172.21.15.31:0/4112789405 conn(0x7f24280d32f0 msgr2=0x7f24280d18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 shutdown_connections 2024-01-19T04:57:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:29.808+0000 7f242f207640 1 -- 172.21.15.31:0/4112789405 wait complete. 2024-01-19T04:57:29.818 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:57:29.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:29.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:29.843 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[57345]: fuse finished with error 0 and tester_r 0 2024-01-19T04:57:29.845 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:57:35.946 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:57:35.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:35.947 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:57:35.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-19T04:57:36.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.120+0000 7fe672410640 1 Processor -- start 2024-01-19T04:57:36.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.120+0000 7fe672410640 1 -- start start 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.120+0000 7fe672410640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe66c160a10 0x7fe66c0b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.120+0000 7fe672410640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0d2e50 0x7fe66c0d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.120+0000 7fe672410640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c160df0 con 0x7fe66c0b4d00 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe672410640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0b7300 con 0x7fe66c160a10 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe672410640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0d1ae0 con 0x7fe66c0d2e50 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe6709ad640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe66c0b4d00 0x7fe66c06b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58936/0 (socket says 172.21.15.31:58936) 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe6709ad640 1 -- 172.21.15.31:0/594326251 learned_addr learned my addr 172.21.15.31:0/594326251 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66bfff640 1 --2- 172.21.15.31:0/594326251 >> v2:172.21.15.137:3300/0 conn(0x7fe66c160a10 0x7fe66c0b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66b7fe640 1 --2- 172.21.15.31:0/594326251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0d2e50 0x7fe66c0d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66affd640 1 -- 172.21.15.31:0/594326251 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513286185 0 0) 0x7fe66c160df0 con 0x7fe66c0b4d00 2024-01-19T04:57:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66affd640 1 -- 172.21.15.31:0/594326251 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe654003520 con 0x7fe66c0b4d00 2024-01-19T04:57:36.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66b7fe640 1 -- 172.21.15.31:0/594326251 >> v2:172.21.15.137:3300/0 conn(0x7fe66c160a10 msgr2=0x7fe66c0b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:36.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66b7fe640 1 --2- 172.21.15.31:0/594326251 >> v2:172.21.15.137:3300/0 conn(0x7fe66c160a10 0x7fe66c0b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66b7fe640 1 -- 172.21.15.31:0/594326251 >> v1:172.21.15.31:6789/0 conn(0x7fe66c0b4d00 legacy=0x7fe66c06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:36.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66b7fe640 1 -- 172.21.15.31:0/594326251 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0d1db0 con 0x7fe66c0d2e50 2024-01-19T04:57:36.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.121+0000 7fe66bfff640 1 --2- 172.21.15.31:0/594326251 >> v2:172.21.15.137:3300/0 conn(0x7fe66c160a10 0x7fe66c0b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.122+0000 7fe66b7fe640 1 --2- 172.21.15.31:0/594326251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0d2e50 0x7fe66c0d15a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6600097a0 tx=0x7fe66002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0020fdd2ebd269 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.122+0000 7fe66affd640 1 -- 172.21.15.31:0/594326251 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660030610 con 0x7fe66c0d2e50 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.122+0000 7fe66affd640 1 -- 172.21.15.31:0/594326251 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660030770 con 0x7fe66c0d2e50 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.122+0000 7fe66affd640 1 -- 172.21.15.31:0/594326251 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660030ae0 con 0x7fe66c0d2e50 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 -- 172.21.15.31:0/594326251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0d2e50 msgr2=0x7fe66c0d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 --2- 172.21.15.31:0/594326251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0d2e50 0x7fe66c0d15a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6600097a0 tx=0x7fe66002b960 comp rx=0 tx=0).stop 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 -- 172.21.15.31:0/594326251 shutdown_connections 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 --2- 172.21.15.31:0/594326251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0d2e50 0x7fe66c0d15a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 --2- 172.21.15.31:0/594326251 >> v2:172.21.15.137:3300/0 conn(0x7fe66c160a10 0x7fe66c0b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 -- 172.21.15.31:0/594326251 >> 172.21.15.31:0/594326251 conn(0x7fe66c072310 msgr2=0x7fe66c0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:36.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 -- 172.21.15.31:0/594326251 shutdown_connections 2024-01-19T04:57:36.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.123+0000 7fe672410640 1 -- 172.21.15.31:0/594326251 wait complete. 2024-01-19T04:57:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.124+0000 7fe672410640 1 Processor -- start 2024-01-19T04:57:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.124+0000 7fe672410640 1 -- start start 2024-01-19T04:57:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe672410640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0b4d00 0x7fe66c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe672410640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe66c0d2e50 0x7fe66c1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe672410640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c160a10 0x7fe66c0cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe672410640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0c6b20 con 0x7fe66c160a10 2024-01-19T04:57:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe672410640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0c6e00 con 0x7fe66c0d2e50 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe672410640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0c70e0 con 0x7fe66c0b4d00 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c160a10 0x7fe66c0cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c160a10 0x7fe66c0cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45662/0 (socket says 172.21.15.31:45662) 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 -- 172.21.15.31:0/3334939032 learned_addr learned my addr 172.21.15.31:0/3334939032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe66b7fe640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe66c0d2e50 0x7fe66c1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe66bfff640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0b4d00 0x7fe66c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 -- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0b4d00 msgr2=0x7fe66c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0b4d00 0x7fe66c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 -- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe66c0d2e50 msgr2=0x7fe66c1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe66c0d2e50 0x7fe66c1567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe6709ad640 1 -- 172.21.15.31:0/3334939032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c009400 con 0x7fe66c160a10 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe66b7fe640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe66c0d2e50 0x7fe66c1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.125+0000 7fe66bfff640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0b4d00 0x7fe66c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.126+0000 7fe6709ad640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c160a10 0x7fe66c0cbc80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe66400cb40 tx=0x7fe66400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.126+0000 7fe668ff9640 1 -- 172.21.15.31:0/3334939032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400aca0 con 0x7fe66c160a10 2024-01-19T04:57:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.126+0000 7fe668ff9640 1 -- 172.21.15.31:0/3334939032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66400ae00 con 0x7fe66c160a10 2024-01-19T04:57:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.126+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe660009400 con 0x7fe66c160a10 2024-01-19T04:57:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.126+0000 7fe668ff9640 1 -- 172.21.15.31:0/3334939032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664012620 con 0x7fe66c160a10 2024-01-19T04:57:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.126+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c0cc1c0 con 0x7fe66c160a10 2024-01-19T04:57:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.127+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c0c7600 con 0x7fe66c160a10 2024-01-19T04:57:36.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.127+0000 7fe668ff9640 1 -- 172.21.15.31:0/3334939032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe664007070 con 0x7fe66c160a10 2024-01-19T04:57:36.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.128+0000 7fe668ff9640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe65806aed0 0x7fe65806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:36.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.128+0000 7fe668ff9640 1 -- 172.21.15.31:0/3334939032 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe664089670 con 0x7fe66c160a10 2024-01-19T04:57:36.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.128+0000 7fe66bfff640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe65806aed0 0x7fe65806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:36.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.128+0000 7fe66bfff640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe65806aed0 0x7fe65806d390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009bb0 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:36.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.132+0000 7fe668ff9640 1 -- 172.21.15.31:0/3334939032 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe66c0c7600 con 0x7fe66c160a10 2024-01-19T04:57:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.347+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe66c0c7600 con 0x7fe66c160a10 2024-01-19T04:57:36.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.347+0000 7fe668ff9640 1 -- 172.21.15.31:0/3334939032 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe66c0c7600 con 0x7fe66c160a10 2024-01-19T04:57:36.347 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-19T04:57:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe65806aed0 msgr2=0x7fe65806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe65806aed0 0x7fe65806d390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009bb0 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2024-01-19T04:57:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c160a10 msgr2=0x7fe66c0cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:36.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c160a10 0x7fe66c0cbc80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe66400cb40 tx=0x7fe66400a060 comp rx=0 tx=0).stop 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 shutdown_connections 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe65806aed0 0x7fe65806d390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c160a10 0x7fe66c0cbc80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe66c0d2e50 0x7fe66c1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 --2- 172.21.15.31:0/3334939032 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe66c0b4d00 0x7fe66c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 >> 172.21.15.31:0/3334939032 conn(0x7fe66c072310 msgr2=0x7fe66c0cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 shutdown_connections 2024-01-19T04:57:36.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:36.349+0000 7fe672410640 1 -- 172.21.15.31:0/3334939032 wait complete. 2024-01-19T04:57:36.359 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-19T04:57:36.360 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:36.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:36.360 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-01-19T04:57:36.375 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-19T04:57:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-19T04:57:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-01-19T04:57:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-01-19T04:57:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42637sec preferred_lft 42637sec 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:36.376 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c57:fcff:feba:867a/64 scope link 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 9a:a9:56:f7:e4:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::98a9:56ff:fef7:e409/64 scope link 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-19T04:57:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 92:b1:87:fd:0d:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::90b1:87ff:fefd:d7a/64 scope link 2024-01-19T04:57:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-01-19T04:57:36.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:36.378 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-19T04:57:36.440 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-01-19T04:57:36.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:36.445 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-01-19T04:57:36.501 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-01-19T04:57:36.501 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-19T04:57:36.501 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-19T04:57:36.501 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-19T04:57:36.501 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-19T04:57:36.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:36.501 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-01-19T04:57:36.501 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-01-19T04:57:36.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-19T04:57:36.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-19T04:57:36.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-19T04:57:36.502 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-01-19T04:57:36.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:36.601 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:57:36.616 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-19T04:57:36.616 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:57:36.616 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:36.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:36.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-01-19T04:57:36.683 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-19T04:57:36.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:57:36.683 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:36.738 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-19T04:57:36.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-19T04:57:36.738 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:36.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:57:36.860 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:57:36.915 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:57:36.915 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:57:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:57:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:36.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:36.918 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:57:36.970 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-19T04:57:36.971 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-19T04:57:37.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-01-19T04:57:37.040 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-01-19T04:57:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-01-19T04:57:37.098 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-0 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-1 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-2 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph/osd/ceph-3 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:37.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:37.100 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-01-19T04:57:37.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-01-19T04:57:37.100+0000 7febc0a45480 -1 init, newargv = 0x563313a36730 newargc=15 2024-01-19T04:57:37.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[58307]: starting ceph client 2024-01-19T04:57:37.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[58307]: starting fuse 2024-01-19T04:57:37.115 INFO:teuthology.orchestra.run.smithi031.stdout:41 2024-01-19T04:57:37.115 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-19T04:57:37.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-19T04:57:37.116 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-01-19T04:57:37.117 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:57:37.236 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.58307.asok 2024-01-19T04:57:37.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-19T04:57:37.246 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58307.asok 2024-01-19T04:57:37.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:37.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58307.asok status 2024-01-19T04:57:37.383 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-01-19T04:57:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "0edf41a622739843d6b978b179ff3227b476dd9d", 2024-01-19T04:57:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-766-g0edf41a6 (0edf41a622739843d6b978b179ff3227b476dd9d) squid (dev)", 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "58307", 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6258, 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-01-19T04:57:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 6258 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 686947332 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 686947332 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.6258 192.168.0.1:0/686947332", 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/686947332", 2024-01-19T04:57:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-01-19T04:57:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 22, 2024-01-19T04:57:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 50, 2024-01-19T04:57:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-01-19T04:57:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-01-19T04:57:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-01-19T04:57:37.386 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:37.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:37.392 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-01-19T04:57:37.392 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:37.407 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-01-19T04:57:37.407 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:37.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:37.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:37.474 DEBUG:teuthology.orchestra.run.smithi031:> stdin-killer -- python3 -c ' 2024-01-19T04:57:37.474 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-01-19T04:57:37.474 DEBUG:teuthology.orchestra.run.smithi031:> import errno 2024-01-19T04:57:37.474 DEBUG:teuthology.orchestra.run.smithi031:> 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> try: 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> if True: 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> ret = os.read(fd, 1024) 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> else: 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> os.write(fd, b'"'"'content'"'"') 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> except OSError as e: 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> if e.errno != errno.EPERM: 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> raise 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> else: 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("client does not check permission of data pool") 2024-01-19T04:57:37.475 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-01-19T04:57:37.582 INFO:teuthology.orchestra.run.smithi031.stderr:Traceback (most recent call last): 2024-01-19T04:57:37.583 INFO:teuthology.orchestra.run.smithi031.stderr: File "", line 15, in 2024-01-19T04:57:37.583 INFO:teuthology.orchestra.run.smithi031.stderr:RuntimeError: client does not check permission of data pool 2024-01-19T04:57:37.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2024-01-19T04:57:37.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:57:37.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-01-19T04:57:37.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.741+0000 7f29297d7640 1 Processor -- start 2024-01-19T04:57:37.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.741+0000 7f29297d7640 1 -- start start 2024-01-19T04:57:37.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f29297d7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f292415b6b0 0x7f29240d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f29297d7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f29240ce090 0x7f29240b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f29297d7640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29240cce50 con 0x7f29240cd2e0 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f29297d7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240d03b0 con 0x7f292415b6b0 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f29297d7640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240cc240 con 0x7f29240ce090 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f2922ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f292415b6b0 0x7f29240d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f2922ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f292415b6b0 0x7f29240d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45674/0 (socket says 172.21.15.31:45674) 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f2922ffd640 1 -- 172.21.15.31:0/2113778403 learned_addr learned my addr 172.21.15.31:0/2113778403 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f29237fe640 1 --2- 172.21.15.31:0/2113778403 >> v2:172.21.15.137:3300/0 conn(0x7f29240ce090 0x7f29240b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f2922ffd640 1 -- 172.21.15.31:0/2113778403 >> v2:172.21.15.137:3300/0 conn(0x7f29240ce090 msgr2=0x7f29240b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f2922ffd640 1 --2- 172.21.15.31:0/2113778403 >> v2:172.21.15.137:3300/0 conn(0x7f29240ce090 0x7f29240b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f2922ffd640 1 -- 172.21.15.31:0/2113778403 >> v1:172.21.15.137:6789/0 conn(0x7f29240cd2e0 legacy=0x7f29240cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f2922ffd640 1 -- 172.21.15.31:0/2113778403 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240cc520 con 0x7f292415b6b0 2024-01-19T04:57:37.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.742+0000 7f29237fe640 1 --2- 172.21.15.31:0/2113778403 >> v2:172.21.15.137:3300/0 conn(0x7f29240ce090 0x7f29240b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:37.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.743+0000 7f2922ffd640 1 --2- 172.21.15.31:0/2113778403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f292415b6b0 0x7f29240d1c20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29240d1090 tx=0x7f291402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802ba3b26163b677 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:37.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.743+0000 7f2921ffb640 1 -- 172.21.15.31:0/2113778403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291402cb70 con 0x7f292415b6b0 2024-01-19T04:57:37.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.743+0000 7f2921ffb640 1 -- 172.21.15.31:0/2113778403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291402ccd0 con 0x7f292415b6b0 2024-01-19T04:57:37.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.743+0000 7f2921ffb640 1 -- 172.21.15.31:0/2113778403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914034610 con 0x7f292415b6b0 2024-01-19T04:57:37.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 -- 172.21.15.31:0/2113778403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f292415b6b0 msgr2=0x7f29240d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 --2- 172.21.15.31:0/2113778403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f292415b6b0 0x7f29240d1c20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29240d1090 tx=0x7f291402bfd0 comp rx=0 tx=0).stop 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 -- 172.21.15.31:0/2113778403 shutdown_connections 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 --2- 172.21.15.31:0/2113778403 >> v2:172.21.15.137:3300/0 conn(0x7f29240ce090 0x7f29240b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 --2- 172.21.15.31:0/2113778403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f292415b6b0 0x7f29240d1c20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 -- 172.21.15.31:0/2113778403 >> 172.21.15.31:0/2113778403 conn(0x7f29240d1100 msgr2=0x7f29240d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 -- 172.21.15.31:0/2113778403 shutdown_connections 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.744+0000 7f29297d7640 1 -- 172.21.15.31:0/2113778403 wait complete. 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.745+0000 7f29297d7640 1 Processor -- start 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.745+0000 7f29297d7640 1 -- start start 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29297d7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f29240cd2e0 0x7f29240b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29297d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29240ce090 0x7f29240c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29297d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f292415b6b0 0x7f29241e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:37.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29297d7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240c7b30 con 0x7f29240ce090 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29297d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240b67a0 con 0x7f292415b6b0 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29297d7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29240c7570 con 0x7f29240cd2e0 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29240ce090 0x7f29240c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29240ce090 0x7f29240c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45690/0 (socket says 172.21.15.31:45690) 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 -- 172.21.15.31:0/434398223 learned_addr learned my addr 172.21.15.31:0/434398223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f2922ffd640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f29240cd2e0 0x7f29240b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 -- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f29240cd2e0 msgr2=0x7f29240b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f29240cd2e0 0x7f29240b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 -- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f292415b6b0 msgr2=0x7f29241e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f292415b6b0 0x7f29241e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 -- 172.21.15.31:0/434398223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914009b30 con 0x7f29240ce090 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f2922ffd640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f29240cd2e0 0x7f29240b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.746+0000 7f29227fc640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29240ce090 0x7f29240c6b70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c430 tx=0x7f290c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.747+0000 7f291b7fe640 1 -- 172.21.15.31:0/434398223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c011790 con 0x7f29240ce090 2024-01-19T04:57:37.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.747+0000 7f291b7fe640 1 -- 172.21.15.31:0/434398223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c0118f0 con 0x7f29240ce090 2024-01-19T04:57:37.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.747+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29240c7850 con 0x7f29240ce090 2024-01-19T04:57:37.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.747+0000 7f291b7fe640 1 -- 172.21.15.31:0/434398223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c011ce0 con 0x7f29240ce090 2024-01-19T04:57:37.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.747+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29240c70b0 con 0x7f29240ce090 2024-01-19T04:57:37.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.748+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29240c7330 con 0x7f29240ce090 2024-01-19T04:57:37.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.748+0000 7f291b7fe640 1 -- 172.21.15.31:0/434398223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f290c01c460 con 0x7f29240ce090 2024-01-19T04:57:37.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.749+0000 7f291b7fe640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f290406aed0 0x7f290406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:37.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.749+0000 7f291b7fe640 1 -- 172.21.15.31:0/434398223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f290c089640 con 0x7f29240ce090 2024-01-19T04:57:37.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.749+0000 7f2922ffd640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f290406aed0 0x7f290406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:37.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.750+0000 7f2922ffd640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f290406aed0 0x7f290406d390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2914035000 tx=0x7f2914030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.753+0000 7f291b7fe640 1 -- 172.21.15.31:0/434398223 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f29240c7330 con 0x7f29240ce090 2024-01-19T04:57:37.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:37.999+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f29240c95f0 con 0x7f29240ce090 2024-01-19T04:57:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.003+0000 7f291b7fe640 1 -- 172.21.15.31:0/434398223 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+141 (secure 0 0 0) 0x7f29240c95f0 con 0x7f29240ce090 2024-01-19T04:57:38.002 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-19T04:57:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB4AKplVWoeGRAAVQQPRtcU9hj5dlxckddwsw== 2024-01-19T04:57:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2024-01-19T04:57:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2024-01-19T04:57:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw pool=cephfs_data" 2024-01-19T04:57:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f290406aed0 msgr2=0x7f290406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f290406aed0 0x7f290406d390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2914035000 tx=0x7f2914030000 comp rx=0 tx=0).stop 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29240ce090 msgr2=0x7f29240c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29240ce090 0x7f29240c6b70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c430 tx=0x7f290c00c8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 shutdown_connections 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f290406aed0 0x7f290406d390 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f292415b6b0 0x7f29241e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29240ce090 0x7f29240c6b70 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 --2- 172.21.15.31:0/434398223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f29240cd2e0 0x7f29240b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.005+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 >> 172.21.15.31:0/434398223 conn(0x7f29240d1100 msgr2=0x7f29240cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.006+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 shutdown_connections 2024-01-19T04:57:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.006+0000 7f29297d7640 1 -- 172.21.15.31:0/434398223 wait complete. 2024-01-19T04:57:38.015 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-19T04:57:38.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:57:38.062 DEBUG:teuthology.orchestra.run.smithi137:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-19T04:57:38.112 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-19T04:57:38.112 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:38.112 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:38.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:38.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.269+0000 7f6a5ff05640 1 Processor -- start 2024-01-19T04:57:38.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.269+0000 7f6a5ff05640 1 -- start start 2024-01-19T04:57:38.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.270+0000 7f6a5ff05640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6a5806e8d0 0x7f6a580d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.270+0000 7f6a5ff05640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580d0040 con 0x7f6a5806ecb0 2024-01-19T04:57:38.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.270+0000 7f6a5ff05640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580cd500 con 0x7f6a580cee00 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.270+0000 7f6a5ff05640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a580cb800 con 0x7f6a5806e8d0 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.270+0000 7f6a5e4a2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6a5806ecb0 0x7f6a580cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58942/0 (socket says 172.21.15.31:58942) 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.270+0000 7f6a5e4a2640 1 -- 172.21.15.31:0/1131300378 learned_addr learned my addr 172.21.15.31:0/1131300378 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.270+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806071198 0 0) 0x7f6a580d0040 con 0x7f6a5806ecb0 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a40003520 con 0x7f6a5806ecb0 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5dca1640 1 --2- 172.21.15.31:0/1131300378 >> v2:172.21.15.137:3300/0 conn(0x7f6a5806e8d0 0x7f6a580d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 912954671 0 0) 0x7f6a40003520 con 0x7f6a5806ecb0 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 >> v2:172.21.15.137:3300/0 conn(0x7f6a5806e8d0 msgr2=0x7f6a580d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 --2- 172.21.15.31:0/1131300378 >> v2:172.21.15.137:3300/0 conn(0x7f6a5806e8d0 0x7f6a580d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 >> v1:172.21.15.137:6789/0 conn(0x7f6a580cee00 legacy=0x7f6a5815c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580cbae0 con 0x7f6a5806ecb0 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f6a480019f0 con 0x7f6a5806ecb0 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 >> v1:172.21.15.31:6789/0 conn(0x7f6a5806ecb0 legacy=0x7f6a580cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a40003970 0x7f6a40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a40005180 0x7f6a4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580cb800 con 0x7f6a400047c0 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4000c120 con 0x7f6a40005180 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4000c2e0 con 0x7f6a40003970 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5d4a0640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5dca1640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a40003970 0x7f6a40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5e4a2640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a40005180 0x7f6a4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5d4a0640 1 -- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a40003970 msgr2=0x7f6a40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5d4a0640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a40003970 0x7f6a40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5d4a0640 1 -- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a40005180 msgr2=0x7f6a4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5d4a0640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a40005180 0x7f6a4000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5d4a0640 1 -- 172.21.15.31:0/1131300378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40003520 con 0x7f6a400047c0 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5e4a2640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a40005180 0x7f6a4000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.271+0000 7f6a5e4a2640 1 -- 172.21.15.31:0/1131300378 reap_dead start 2024-01-19T04:57:38.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.272+0000 7f6a5d4a0640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c001720 tx=0x7f6a400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a172ef6efbc1b5e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.272+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0052d0 con 0x7f6a400047c0 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.272+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c02e050 con 0x7f6a400047c0 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.272+0000 7f6a5cc9f640 1 -- 172.21.15.31:0/1131300378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c003830 con 0x7f6a400047c0 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.273+0000 7f6a5ff05640 1 -- 172.21.15.31:0/1131300378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a400047c0 msgr2=0x7f6a40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.273+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c001720 tx=0x7f6a400057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.273+0000 7f6a5ff05640 1 -- 172.21.15.31:0/1131300378 shutdown_connections 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.273+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/1131300378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.273+0000 7f6a5ff05640 1 -- 172.21.15.31:0/1131300378 >> 172.21.15.31:0/1131300378 conn(0x7f6a580b2c60 msgr2=0x7f6a580b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.273+0000 7f6a5ff05640 1 -- 172.21.15.31:0/1131300378 shutdown_connections 2024-01-19T04:57:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.273+0000 7f6a5ff05640 1 -- 172.21.15.31:0/1131300378 wait complete. 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.274+0000 7f6a5ff05640 1 Processor -- start 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.274+0000 7f6a5ff05640 1 -- start start 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a400047c0 0x7f6a5814e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a580b4da0 0x7f6a5814d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a5814fbf0 0x7f6a580b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5806b6c0 con 0x7f6a5814fbf0 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5806d490 con 0x7f6a400047c0 2024-01-19T04:57:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580b5260 con 0x7f6a580b4da0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a5814fbf0 0x7f6a580b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a5814fbf0 0x7f6a580b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45704/0 (socket says 172.21.15.31:45704) 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 -- 172.21.15.31:0/2328776253 learned_addr learned my addr 172.21.15.31:0/2328776253 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5dca1640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a400047c0 0x7f6a5814e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5d4a0640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a580b4da0 0x7f6a5814d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 -- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a580b4da0 msgr2=0x7f6a5814d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a580b4da0 0x7f6a5814d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 -- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a400047c0 msgr2=0x7f6a5814e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a400047c0 0x7f6a5814e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 -- 172.21.15.31:0/2328776253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580b5540 con 0x7f6a5814fbf0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5e4a2640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a5814fbf0 0x7f6a580b3040 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6a48001580 tx=0x7f6a4800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a567fc640 1 -- 172.21.15.31:0/2328776253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48004d60 con 0x7f6a5814fbf0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a567fc640 1 -- 172.21.15.31:0/2328776253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4800bb80 con 0x7f6a5814fbf0 2024-01-19T04:57:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a580b5820 con 0x7f6a5814fbf0 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a567fc640 1 -- 172.21.15.31:0/2328776253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48007070 con 0x7f6a5814fbf0 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.275+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5814e720 con 0x7f6a5814fbf0 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.276+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a58074a80 con 0x7f6a5814fbf0 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.277+0000 7f6a567fc640 1 -- 172.21.15.31:0/2328776253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6a480071d0 con 0x7f6a5814fbf0 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.277+0000 7f6a567fc640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6a4406aed0 0x7f6a4406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.277+0000 7f6a567fc640 1 -- 172.21.15.31:0/2328776253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a48089880 con 0x7f6a5814fbf0 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.277+0000 7f6a5dca1640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6a4406aed0 0x7f6a4406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.278+0000 7f6a5dca1640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6a4406aed0 0x7f6a4406d390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009b40 tx=0x7f6a50008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.281+0000 7f6a567fc640 1 -- 172.21.15.31:0/2328776253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6a48054800 con 0x7f6a5814fbf0 2024-01-19T04:57:38.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.461+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a5806b800 con 0x7f6a5814fbf0 2024-01-19T04:57:38.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.462+0000 7f6a567fc640 1 -- 172.21.15.31:0/2328776253 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f6a5806b800 con 0x7f6a5814fbf0 2024-01-19T04:57:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:38.462 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.463+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6a4406aed0 msgr2=0x7f6a4406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.463+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6a4406aed0 0x7f6a4406d390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009b40 tx=0x7f6a50008040 comp rx=0 tx=0).stop 2024-01-19T04:57:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.463+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a5814fbf0 msgr2=0x7f6a580b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.463+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a5814fbf0 0x7f6a580b3040 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6a48001580 tx=0x7f6a4800b4a0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 shutdown_connections 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6a4406aed0 0x7f6a4406d390 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a5814fbf0 0x7f6a580b3040 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a580b4da0 0x7f6a5814d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 --2- 172.21.15.31:0/2328776253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a400047c0 0x7f6a5814e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 >> 172.21.15.31:0/2328776253 conn(0x7f6a580b2c60 msgr2=0x7f6a580c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 shutdown_connections 2024-01-19T04:57:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.464+0000 7f6a5ff05640 1 -- 172.21.15.31:0/2328776253 wait complete. 2024-01-19T04:57:38.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:57:10.830975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:57:11.850371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:38.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.581+0000 7f0ad0456640 1 Processor -- start 2024-01-19T04:57:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.581+0000 7f0ad0456640 1 -- start start 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0ad0456640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac80cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0ad0456640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ac80cc490 0x7f0ac806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0ad0456640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac80c6130 con 0x7f0ac806aaf0 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0ad0456640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac80cc910 con 0x7f0ac80cc490 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0ad0456640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac806af80 con 0x7f0ac815d340 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0acd9f1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ac80cc490 0x7f0ac806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0acd9f1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ac80cc490 0x7f0ac806a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45714/0 (socket says 172.21.15.31:45714) 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0acd9f1640 1 -- 172.21.15.31:0/182641500 learned_addr learned my addr 172.21.15.31:0/182641500 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0ace1f2640 1 --2- 172.21.15.31:0/182641500 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac80cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0acd9f1640 1 -- 172.21.15.31:0/182641500 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 msgr2=0x7f0ac80cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0acd9f1640 1 --2- 172.21.15.31:0/182641500 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac80cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0acd9f1640 1 -- 172.21.15.31:0/182641500 >> v1:172.21.15.137:6789/0 conn(0x7f0ac806aaf0 legacy=0x7f0ac8160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.582+0000 7f0acd9f1640 1 -- 172.21.15.31:0/182641500 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80ce630 con 0x7f0ac80cc490 2024-01-19T04:57:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.583+0000 7f0acd9f1640 1 --2- 172.21.15.31:0/182641500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac80cc490 0x7f0ac806a5b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80098a0 tx=0x7f0ab802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d01f3dae27103 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.583+0000 7f0acd1f0640 1 -- 172.21.15.31:0/182641500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8031520 con 0x7f0ac80cc490 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.583+0000 7f0acd1f0640 1 -- 172.21.15.31:0/182641500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8031680 con 0x7f0ac80cc490 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.583+0000 7f0acd1f0640 1 -- 172.21.15.31:0/182641500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8031a10 con 0x7f0ac80cc490 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 -- 172.21.15.31:0/182641500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac80cc490 msgr2=0x7f0ac806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 --2- 172.21.15.31:0/182641500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac80cc490 0x7f0ac806a5b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80098a0 tx=0x7f0ab802b7c0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 -- 172.21.15.31:0/182641500 shutdown_connections 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 --2- 172.21.15.31:0/182641500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac80cc490 0x7f0ac806a5b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 --2- 172.21.15.31:0/182641500 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac80cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 -- 172.21.15.31:0/182641500 >> 172.21.15.31:0/182641500 conn(0x7f0ac80b1c50 msgr2=0x7f0ac80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 -- 172.21.15.31:0/182641500 shutdown_connections 2024-01-19T04:57:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.584+0000 7f0ad0456640 1 -- 172.21.15.31:0/182641500 wait complete. 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.585+0000 7f0ad0456640 1 Processor -- start 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.585+0000 7f0ad0456640 1 -- start start 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ad0456640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac806aaf0 0x7f0ac8149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ad0456640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac80cc490 0x7f0ac8149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ad0456640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ad0456640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80b5b60 con 0x7f0ac806aaf0 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ad0456640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815ba50 con 0x7f0ac80cc490 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ad0456640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815bd30 con 0x7f0ac815d340 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac806aaf0 0x7f0ac8149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac806aaf0 0x7f0ac8149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45724/0 (socket says 172.21.15.31:45724) 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 -- 172.21.15.31:0/3598447190 learned_addr learned my addr 172.21.15.31:0/3598447190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace9f3640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0acd9f1640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac80cc490 0x7f0ac8149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 -- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 msgr2=0x7f0ac815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 -- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac80cc490 msgr2=0x7f0ac8149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac80cc490 0x7f0ac8149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 -- 172.21.15.31:0/3598447190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80094d0 con 0x7f0ac806aaf0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace9f3640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ace1f2640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac806aaf0 0x7f0ac8149690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400d5f0 tx=0x7f0ac400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0abeffd640 1 -- 172.21.15.31:0/3598447190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40125f0 con 0x7f0ac806aaf0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0abeffd640 1 -- 172.21.15.31:0/3598447190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac4012750 con 0x7f0ac806aaf0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac815c010 con 0x7f0ac806aaf0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.586+0000 7f0abeffd640 1 -- 172.21.15.31:0/3598447190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4012b00 con 0x7f0ac806aaf0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.587+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac8156280 con 0x7f0ac806aaf0 2024-01-19T04:57:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.588+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8074900 con 0x7f0ac806aaf0 2024-01-19T04:57:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.588+0000 7f0abeffd640 1 -- 172.21.15.31:0/3598447190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0ac4012c60 con 0x7f0ac806aaf0 2024-01-19T04:57:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.588+0000 7f0abeffd640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0aac06aed0 0x7f0aac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.588+0000 7f0abeffd640 1 -- 172.21.15.31:0/3598447190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ac4089bd0 con 0x7f0ac806aaf0 2024-01-19T04:57:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.589+0000 7f0acd9f1640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0aac06aed0 0x7f0aac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.589+0000 7f0acd9f1640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0aac06aed0 0x7f0aac06d390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8035000 tx=0x7f0ab8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.593+0000 7f0abeffd640 1 -- 172.21.15.31:0/3598447190 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0ac4054a10 con 0x7f0ac806aaf0 2024-01-19T04:57:38.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.768+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ac80cd870 con 0x7f0ac806aaf0 2024-01-19T04:57:38.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.768+0000 7f0abeffd640 1 -- 172.21.15.31:0/3598447190 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ac4058030 con 0x7f0ac806aaf0 2024-01-19T04:57:38.768 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0aac06aed0 msgr2=0x7f0aac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0aac06aed0 0x7f0aac06d390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8035000 tx=0x7f0ab8030000 comp rx=0 tx=0).stop 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac806aaf0 msgr2=0x7f0ac8149690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac806aaf0 0x7f0ac8149690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400d5f0 tx=0x7f0ac400dab0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 shutdown_connections 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0aac06aed0 0x7f0aac06d390 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ac815d340 0x7f0ac815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac80cc490 0x7f0ac8149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 --2- 172.21.15.31:0/3598447190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac806aaf0 0x7f0ac8149690 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 >> 172.21.15.31:0/3598447190 conn(0x7f0ac80b1c50 msgr2=0x7f0ac80c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:38.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 shutdown_connections 2024-01-19T04:57:38.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.770+0000 7f0ad0456640 1 -- 172.21.15.31:0/3598447190 wait complete. 2024-01-19T04:57:38.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:57:38.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.885+0000 7f98591c5640 1 Processor -- start 2024-01-19T04:57:38.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.885+0000 7f98591c5640 1 -- start start 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f98591c5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98540ceff0 0x7f985415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f98591c5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98540d1600 con 0x7f98540d3e50 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f98591c5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985415dee0 con 0x7f98540d18e0 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f98591c5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540cf460 con 0x7f98540ceff0 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f985359e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98540ceff0 0x7f985415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f985259c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f98540d3e50 0x7f98540d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58952/0 (socket says 172.21.15.31:58952) 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f985259c640 1 -- 172.21.15.31:0/1969926352 learned_addr learned my addr 172.21.15.31:0/1969926352 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f985359e640 1 -- 172.21.15.31:0/1969926352 >> v1:172.21.15.137:6789/0 conn(0x7f98540d18e0 legacy=0x7f98540d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f985359e640 1 -- 172.21.15.31:0/1969926352 >> v1:172.21.15.31:6789/0 conn(0x7f98540d3e50 legacy=0x7f98540d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f985359e640 1 -- 172.21.15.31:0/1969926352 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985415e1c0 con 0x7f98540ceff0 2024-01-19T04:57:38.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.886+0000 7f9851d9b640 1 -- 172.21.15.31:0/1969926352 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033538890 0 0) 0x7f98540d1600 con 0x7f98540d3e50 2024-01-19T04:57:38.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.887+0000 7f985359e640 1 --2- 172.21.15.31:0/1969926352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540ceff0 0x7f985415d860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9844009d90 tx=0x7f984402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78eed440aba86215 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.887+0000 7f9851d9b640 1 -- 172.21.15.31:0/1969926352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984402cc60 con 0x7f98540ceff0 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.887+0000 7f9851d9b640 1 -- 172.21.15.31:0/1969926352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984402cdc0 con 0x7f98540ceff0 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.887+0000 7f9851d9b640 1 -- 172.21.15.31:0/1969926352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844034660 con 0x7f98540ceff0 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.888+0000 7f98591c5640 1 -- 172.21.15.31:0/1969926352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540ceff0 msgr2=0x7f985415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.888+0000 7f98591c5640 1 --2- 172.21.15.31:0/1969926352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540ceff0 0x7f985415d860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9844009d90 tx=0x7f984402beb0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.888+0000 7f98591c5640 1 -- 172.21.15.31:0/1969926352 shutdown_connections 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.888+0000 7f98591c5640 1 --2- 172.21.15.31:0/1969926352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540ceff0 0x7f985415d860 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.888+0000 7f98591c5640 1 -- 172.21.15.31:0/1969926352 >> 172.21.15.31:0/1969926352 conn(0x7f985409c600 msgr2=0x7f98540d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.888+0000 7f98591c5640 1 -- 172.21.15.31:0/1969926352 shutdown_connections 2024-01-19T04:57:38.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.888+0000 7f98591c5640 1 -- 172.21.15.31:0/1969926352 wait complete. 2024-01-19T04:57:38.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.889+0000 7f98591c5640 1 Processor -- start 2024-01-19T04:57:38.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.889+0000 7f98591c5640 1 -- start start 2024-01-19T04:57:38.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f98591c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98540ceff0 0x7f9854155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f98591c5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98540d18e0 0x7f98540c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f98591c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540d3e50 0x7f98540c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f98591c5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985415a390 con 0x7f98540d3e50 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f98591c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541578f0 con 0x7f98540ceff0 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f98591c5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9854157bd0 con 0x7f98540d18e0 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540d3e50 0x7f98540c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540d3e50 0x7f98540c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45750/0 (socket says 172.21.15.31:45750) 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 -- 172.21.15.31:0/2432669613 learned_addr learned my addr 172.21.15.31:0/2432669613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985259c640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98540d18e0 0x7f98540c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f9852d9d640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98540ceff0 0x7f9854155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 -- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98540d18e0 msgr2=0x7f98540c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98540d18e0 0x7f98540c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 -- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98540ceff0 msgr2=0x7f9854155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98540ceff0 0x7f9854155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 -- 172.21.15.31:0/2432669613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844009a70 con 0x7f98540d3e50 2024-01-19T04:57:38.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f9852d9d640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98540ceff0 0x7f9854155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:38.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.890+0000 7f985359e640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540d3e50 0x7f98540c8d20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9844009d90 tx=0x7f9844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.891+0000 7f983b7fe640 1 -- 172.21.15.31:0/2432669613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984403e610 con 0x7f98540d3e50 2024-01-19T04:57:38.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.891+0000 7f983b7fe640 1 -- 172.21.15.31:0/2432669613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984403e790 con 0x7f98540d3e50 2024-01-19T04:57:38.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.891+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9854157eb0 con 0x7f98540d3e50 2024-01-19T04:57:38.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.891+0000 7f983b7fe640 1 -- 172.21.15.31:0/2432669613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984403eb40 con 0x7f98540d3e50 2024-01-19T04:57:38.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.891+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9854155f40 con 0x7f98540d3e50 2024-01-19T04:57:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.892+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985415a840 con 0x7f98540d3e50 2024-01-19T04:57:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.892+0000 7f983b7fe640 1 -- 172.21.15.31:0/2432669613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f984403b050 con 0x7f98540d3e50 2024-01-19T04:57:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.892+0000 7f983b7fe640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f984006aed0 0x7f984006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.893+0000 7f983b7fe640 1 -- 172.21.15.31:0/2432669613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98440abff0 con 0x7f98540d3e50 2024-01-19T04:57:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.893+0000 7f9852d9d640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f984006aed0 0x7f984006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:38.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.893+0000 7f9852d9d640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f984006aed0 0x7f984006d390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9854159450 tx=0x7f984c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:38.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:38.897+0000 7f983b7fe640 1 -- 172.21.15.31:0/2432669613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f985415a840 con 0x7f98540d3e50 2024-01-19T04:57:39.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.073+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f985415a840 con 0x7f98540d3e50 2024-01-19T04:57:39.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.073+0000 7f983b7fe640 1 -- 172.21.15.31:0/2432669613 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9844039020 con 0x7f98540d3e50 2024-01-19T04:57:39.073 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f984006aed0 msgr2=0x7f984006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f984006aed0 0x7f984006d390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9854159450 tx=0x7f984c009040 comp rx=0 tx=0).stop 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540d3e50 msgr2=0x7f98540c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540d3e50 0x7f98540c8d20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9844009d90 tx=0x7f9844004a40 comp rx=0 tx=0).stop 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 shutdown_connections 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f984006aed0 0x7f984006d390 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98540d3e50 0x7f98540c8d20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98540d18e0 0x7f98540c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 --2- 172.21.15.31:0/2432669613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98540ceff0 0x7f9854155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 >> 172.21.15.31:0/2432669613 conn(0x7f985409c600 msgr2=0x7f98540d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:39.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 shutdown_connections 2024-01-19T04:57:39.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.075+0000 7f98591c5640 1 -- 172.21.15.31:0/2432669613 wait complete. 2024-01-19T04:57:39.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:57:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.190+0000 7f61221be640 1 Processor -- start 2024-01-19T04:57:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.190+0000 7f61221be640 1 -- start start 2024-01-19T04:57:39.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f61221be640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d18e0 0x7f611c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f61221be640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f611c0d0ef0 0x7f611c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f61221be640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611c15dee0 con 0x7f611c0d3e50 2024-01-19T04:57:39.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f61221be640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611c15e1c0 con 0x7f611c0d0ef0 2024-01-19T04:57:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f61221be640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c15e4a0 con 0x7f611c0d18e0 2024-01-19T04:57:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f611bfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f611c0d0ef0 0x7f611c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f611b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d18e0 0x7f611c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f611b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d18e0 0x7f611c15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:45140/0 (socket says 172.21.15.31:45140) 2024-01-19T04:57:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f611b7fe640 1 -- 172.21.15.31:0/3035434622 learned_addr learned my addr 172.21.15.31:0/3035434622 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f611bfff640 1 -- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d18e0 msgr2=0x7f611c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f611bfff640 1 --2- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d18e0 0x7f611c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.191+0000 7f611bfff640 1 -- 172.21.15.31:0/3035434622 >> v1:172.21.15.137:6789/0 conn(0x7f611c0d3e50 legacy=0x7f611c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.192+0000 7f611bfff640 1 -- 172.21.15.31:0/3035434622 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c15e780 con 0x7f611c0d0ef0 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.192+0000 7f611a7fc640 1 -- 172.21.15.31:0/3035434622 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211372621 0 0) 0x7f611c15dee0 con 0x7f611c0d3e50 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.192+0000 7f611b7fe640 1 --2- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d18e0 0x7f611c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.192+0000 7f611bfff640 1 --2- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c0cdfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6114009d90 tx=0x7f611402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ffc10f01b8a149c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f611a7fc640 1 -- 172.21.15.31:0/3035434622 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611402cc50 con 0x7f611c0d0ef0 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f611a7fc640 1 -- 172.21.15.31:0/3035434622 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611402cdb0 con 0x7f611c0d0ef0 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f611a7fc640 1 -- 172.21.15.31:0/3035434622 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114034660 con 0x7f611c0d0ef0 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 -- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 msgr2=0x7f611c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 --2- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c0cdfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6114009d90 tx=0x7f611402bea0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 -- 172.21.15.31:0/3035434622 shutdown_connections 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 --2- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c0cdfe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 --2- 172.21.15.31:0/3035434622 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d18e0 0x7f611c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 -- 172.21.15.31:0/3035434622 >> 172.21.15.31:0/3035434622 conn(0x7f611c09c600 msgr2=0x7f611c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 -- 172.21.15.31:0/3035434622 shutdown_connections 2024-01-19T04:57:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.193+0000 7f61221be640 1 -- 172.21.15.31:0/3035434622 wait complete. 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.194+0000 7f61221be640 1 Processor -- start 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.194+0000 7f61221be640 1 -- start start 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f61221be640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c1f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f61221be640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611c0d18e0 0x7f611c0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f61221be640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d3e50 0x7f611c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f61221be640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0c9560 con 0x7f611c0d18e0 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f61221be640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0cabe0 con 0x7f611c0d0ef0 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f61221be640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0c9280 con 0x7f611c0d3e50 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f611affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611c0d18e0 0x7f611c0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f611affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611c0d18e0 0x7f611c0c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45766/0 (socket says 172.21.15.31:45766) 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f611affd640 1 -- 172.21.15.31:0/529591700 learned_addr learned my addr 172.21.15.31:0/529591700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f611bfff640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d3e50 0x7f611c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.195+0000 7f611b7fe640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c1f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f611affd640 1 -- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d3e50 msgr2=0x7f611c0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f611affd640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d3e50 0x7f611c0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f611affd640 1 -- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 msgr2=0x7f611c1f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f611affd640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c1f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f611affd640 1 -- 172.21.15.31:0/529591700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61100094d0 con 0x7f611c0d18e0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f611b7fe640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c1f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f611affd640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611c0d18e0 0x7f611c0c8be0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c410 tx=0x7f610c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f6103fff640 1 -- 172.21.15.31:0/529591700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c0035f0 con 0x7f611c0d18e0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f6103fff640 1 -- 172.21.15.31:0/529591700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c003770 con 0x7f611c0d18e0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6114009a70 con 0x7f611c0d18e0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f6103fff640 1 -- 172.21.15.31:0/529591700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c003b20 con 0x7f611c0d18e0 2024-01-19T04:57:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.196+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c0ca870 con 0x7f611c0d18e0 2024-01-19T04:57:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.197+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c0ca3d0 con 0x7f611c0d18e0 2024-01-19T04:57:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.197+0000 7f6103fff640 1 -- 172.21.15.31:0/529591700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f610c003c80 con 0x7f611c0d18e0 2024-01-19T04:57:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.198+0000 7f6103fff640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f610806aed0 0x7f610806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.198+0000 7f6103fff640 1 -- 172.21.15.31:0/529591700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f610c088bf0 con 0x7f611c0d18e0 2024-01-19T04:57:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.198+0000 7f611b7fe640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f610806aed0 0x7f610806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.199+0000 7f611b7fe640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f610806aed0 0x7f610806d390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6110009db0 tx=0x7f6110008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.202+0000 7f6103fff640 1 -- 172.21.15.31:0/529591700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f610c053b70 con 0x7f611c0d18e0 2024-01-19T04:57:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.382+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f611c0cad50 con 0x7f611c0d18e0 2024-01-19T04:57:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.383+0000 7f6103fff640 1 -- 172.21.15.31:0/529591700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f610c017020 con 0x7f611c0d18e0 2024-01-19T04:57:39.382 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f610806aed0 msgr2=0x7f610806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f610806aed0 0x7f610806d390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6110009db0 tx=0x7f6110008040 comp rx=0 tx=0).stop 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611c0d18e0 msgr2=0x7f611c0c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611c0d18e0 0x7f611c0c8be0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c410 tx=0x7f610c00c8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 shutdown_connections 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f610806aed0 0x7f610806d390 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f611c0d3e50 0x7f611c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611c0d18e0 0x7f611c0c8be0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 --2- 172.21.15.31:0/529591700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f611c0d0ef0 0x7f611c1f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.384+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 >> 172.21.15.31:0/529591700 conn(0x7f611c09c600 msgr2=0x7f611c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.385+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 shutdown_connections 2024-01-19T04:57:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.385+0000 7f61221be640 1 -- 172.21.15.31:0/529591700 wait complete. 2024-01-19T04:57:39.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-19T04:57:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.497+0000 7f8208631640 1 Processor -- start 2024-01-19T04:57:39.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.497+0000 7f8208631640 1 -- start start 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8208631640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f820006acc0 0x7f8200168170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8208631640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82000c6130 con 0x7f820015d340 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8208631640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82000ce630 con 0x7f82000cc660 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8208631640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82000c5520 con 0x7f820006acc0 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8206bce640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f820006acc0 0x7f8200168170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f82063cd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f820015d340 0x7f82000cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39470/0 (socket says 172.21.15.31:39470) 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f82063cd640 1 -- 172.21.15.31:0/1473286389 learned_addr learned my addr 172.21.15.31:0/1473286389 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8206bce640 1 -- 172.21.15.31:0/1473286389 >> v1:172.21.15.137:6789/0 conn(0x7f82000cc660 legacy=0x7f820006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8206bce640 1 -- 172.21.15.31:0/1473286389 >> v1:172.21.15.31:6789/0 conn(0x7f820015d340 legacy=0x7f82000cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.498+0000 7f8206bce640 1 -- 172.21.15.31:0/1473286389 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82000c5800 con 0x7f820006acc0 2024-01-19T04:57:39.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.499+0000 7f8206bce640 1 --2- 172.21.15.31:0/1473286389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820006acc0 0x7f8200168170 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009d90 tx=0x7f81f802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de813de7f40d11c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:39.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.499+0000 7f82053cb640 1 -- 172.21.15.31:0/1473286389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f802cbd0 con 0x7f820006acc0 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.499+0000 7f82053cb640 1 -- 172.21.15.31:0/1473286389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f802cd30 con 0x7f820006acc0 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.499+0000 7f82053cb640 1 -- 172.21.15.31:0/1473286389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f802cea0 con 0x7f820006acc0 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.500+0000 7f8208631640 1 -- 172.21.15.31:0/1473286389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820006acc0 msgr2=0x7f8200168170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.500+0000 7f8208631640 1 --2- 172.21.15.31:0/1473286389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820006acc0 0x7f8200168170 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009d90 tx=0x7f81f802be20 comp rx=0 tx=0).stop 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.500+0000 7f8208631640 1 -- 172.21.15.31:0/1473286389 shutdown_connections 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.500+0000 7f8208631640 1 --2- 172.21.15.31:0/1473286389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820006acc0 0x7f8200168170 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.500+0000 7f8208631640 1 -- 172.21.15.31:0/1473286389 >> 172.21.15.31:0/1473286389 conn(0x7f82000b1c50 msgr2=0x7f82000b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.500+0000 7f8208631640 1 -- 172.21.15.31:0/1473286389 shutdown_connections 2024-01-19T04:57:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.500+0000 7f8208631640 1 -- 172.21.15.31:0/1473286389 wait complete. 2024-01-19T04:57:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.501+0000 7f8208631640 1 Processor -- start 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.501+0000 7f8208631640 1 -- start start 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8208631640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f820006acc0 0x7f8200154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8208631640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82000cc660 0x7f82000ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8208631640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820015d340 0x7f8200071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8208631640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820006e510 con 0x7f820015d340 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8208631640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8200154980 con 0x7f820006acc0 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8208631640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820006e230 con 0x7f82000cc660 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820015d340 0x7f8200071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820015d340 0x7f8200071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54400/0 (socket says 172.21.15.31:54400) 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 -- 172.21.15.31:0/192948906 learned_addr learned my addr 172.21.15.31:0/192948906 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8205bcc640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82000cc660 0x7f82000ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f82063cd640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f820006acc0 0x7f8200154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 -- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82000cc660 msgr2=0x7f82000ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82000cc660 0x7f82000ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 -- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f820006acc0 msgr2=0x7f8200154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f820006acc0 0x7f8200154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 -- 172.21.15.31:0/192948906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009a70 con 0x7f820015d340 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f8206bce640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820015d340 0x7f8200071340 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f81f8037040 tx=0x7f81f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.502+0000 7f81f6ffd640 1 -- 172.21.15.31:0/192948906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f802cea0 con 0x7f820015d340 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.503+0000 7f81f6ffd640 1 -- 172.21.15.31:0/192948906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8040650 con 0x7f820015d340 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.503+0000 7f81f6ffd640 1 -- 172.21.15.31:0/192948906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8040a00 con 0x7f820015d340 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.503+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82000c8470 con 0x7f820015d340 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.503+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82000c8a30 con 0x7f820015d340 2024-01-19T04:57:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.503+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8200159600 con 0x7f820015d340 2024-01-19T04:57:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.504+0000 7f81f6ffd640 1 -- 172.21.15.31:0/192948906 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f81f8040b60 con 0x7f820015d340 2024-01-19T04:57:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.504+0000 7f81f6ffd640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f81e406aed0 0x7f81e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.504+0000 7f81f6ffd640 1 -- 172.21.15.31:0/192948906 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81f80ac440 con 0x7f820015d340 2024-01-19T04:57:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.504+0000 7f82063cd640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f81e406aed0 0x7f81e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.505+0000 7f82063cd640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f81e406aed0 0x7f81e406d390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005880 tx=0x7f81fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:39.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.509+0000 7f81f6ffd640 1 -- 172.21.15.31:0/192948906 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f81f80773a0 con 0x7f820015d340 2024-01-19T04:57:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.700+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f82000c82d0 con 0x7f820015d340 2024-01-19T04:57:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.700+0000 7f81f6ffd640 1 -- 172.21.15.31:0/192948906 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f82000c82d0 con 0x7f820015d340 2024-01-19T04:57:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-01-19T04:57:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f81e406aed0 msgr2=0x7f81e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f81e406aed0 0x7f81e406d390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005880 tx=0x7f81fc009040 comp rx=0 tx=0).stop 2024-01-19T04:57:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820015d340 msgr2=0x7f8200071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820015d340 0x7f8200071340 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f81f8037040 tx=0x7f81f8004a40 comp rx=0 tx=0).stop 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 shutdown_connections 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f81e406aed0 0x7f81e406d390 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f820015d340 0x7f8200071340 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82000cc660 0x7f82000ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 --2- 172.21.15.31:0/192948906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f820006acc0 0x7f8200154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 >> 172.21.15.31:0/192948906 conn(0x7f82000b1c50 msgr2=0x7f82000cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 shutdown_connections 2024-01-19T04:57:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:39.703+0000 7f8208631640 1 -- 172.21.15.31:0/192948906 wait complete. 2024-01-19T04:57:39.713 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-01-19T04:57:39.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:39.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-19T04:57:39.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[58307]: fuse finished with error 0 and tester_r 0 2024-01-19T04:57:39.739 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:57:39.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:39.739 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-19T04:57:39.890 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:57:39.891 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:39.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:57:39.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:57:39.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:39.910 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-19T04:57:39.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:57:39.969 INFO:teuthology.orchestra.run.smithi137.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-01-19T04:57:39.969 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:39.969 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:39.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057151,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20760 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249472k,nr_inodes=812368,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-19T04:57:39.984 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:57:39.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:39.984 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-19T04:57:40.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:57:40.039 INFO:teuthology.orchestra.run.smithi031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-19T04:57:40.039 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:57:40.039 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:40.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:57:40.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:57:40.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:40.058 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-19T04:57:40.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:57:40.116 INFO:teuthology.orchestra.run.smithi137.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-19T04:57:40.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:57:40.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.218+0000 7f8363a2b640 1 Processor -- start 2024-01-19T04:57:40.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.218+0000 7f8363a2b640 1 -- start start 2024-01-19T04:57:40.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f8363a2b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f835c15d8a0 0x7f835c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f8363a2b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835c06bf30 con 0x7f835c15f0c0 2024-01-19T04:57:40.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f8363a2b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835c0cb9e0 con 0x7f835c15eb70 2024-01-19T04:57:40.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f8363a2b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c0cbcc0 con 0x7f835c15d8a0 2024-01-19T04:57:40.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f8361fc8640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f835c15f0c0 0x7f835c0b2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39474/0 (socket says 172.21.15.31:39474) 2024-01-19T04:57:40.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f8361fc8640 1 -- 172.21.15.31:0/1590916746 learned_addr learned my addr 172.21.15.31:0/1590916746 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:40.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f83617c7640 1 --2- 172.21.15.31:0/1590916746 >> v2:172.21.15.137:3300/0 conn(0x7f835c15d8a0 0x7f835c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f834ffff640 1 -- 172.21.15.31:0/1590916746 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673549548 0 0) 0x7f835c06bf30 con 0x7f835c15f0c0 2024-01-19T04:57:40.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.219+0000 7f834ffff640 1 -- 172.21.15.31:0/1590916746 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8348003520 con 0x7f835c15f0c0 2024-01-19T04:57:40.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.220+0000 7f83617c7640 1 -- 172.21.15.31:0/1590916746 >> v1:172.21.15.137:6789/0 conn(0x7f835c15eb70 legacy=0x7f835c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.220+0000 7f83617c7640 1 -- 172.21.15.31:0/1590916746 >> v1:172.21.15.31:6789/0 conn(0x7f835c15f0c0 legacy=0x7f835c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.220+0000 7f834ffff640 1 -- 172.21.15.31:0/1590916746 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2853581755 0 0) 0x7f8348003520 con 0x7f835c15f0c0 2024-01-19T04:57:40.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.220+0000 7f83617c7640 1 -- 172.21.15.31:0/1590916746 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0cbfa0 con 0x7f835c15d8a0 2024-01-19T04:57:40.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.220+0000 7f83617c7640 1 --2- 172.21.15.31:0/1590916746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c06b6c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83500097a0 tx=0x7f835002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=784dae2b86965eb8 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f834ffff640 1 -- 172.21.15.31:0/1590916746 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350031830 con 0x7f835c15d8a0 2024-01-19T04:57:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f834ffff640 1 -- 172.21.15.31:0/1590916746 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8350031990 con 0x7f835c15d8a0 2024-01-19T04:57:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f834ffff640 1 -- 172.21.15.31:0/1590916746 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350031d20 con 0x7f835c15d8a0 2024-01-19T04:57:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f8363a2b640 1 -- 172.21.15.31:0/1590916746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 msgr2=0x7f835c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f8363a2b640 1 --2- 172.21.15.31:0/1590916746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c06b6c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83500097a0 tx=0x7f835002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f8363a2b640 1 -- 172.21.15.31:0/1590916746 shutdown_connections 2024-01-19T04:57:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f8363a2b640 1 --2- 172.21.15.31:0/1590916746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c06b6c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f8363a2b640 1 -- 172.21.15.31:0/1590916746 >> 172.21.15.31:0/1590916746 conn(0x7f835c06abe0 msgr2=0x7f835c066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f8363a2b640 1 -- 172.21.15.31:0/1590916746 shutdown_connections 2024-01-19T04:57:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.221+0000 7f8363a2b640 1 -- 172.21.15.31:0/1590916746 wait complete. 2024-01-19T04:57:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.222+0000 7f8363a2b640 1 Processor -- start 2024-01-19T04:57:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.222+0000 7f8363a2b640 1 -- start start 2024-01-19T04:57:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8363a2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8363a2b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f835c15eb70 0x7f835c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8363a2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835c15f0c0 0x7f835c150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8363a2b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0aea90 con 0x7f835c15f0c0 2024-01-19T04:57:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8363a2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0aed70 con 0x7f835c15d8a0 2024-01-19T04:57:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8363a2b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0af050 con 0x7f835c15eb70 2024-01-19T04:57:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8361fc8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835c15f0c0 0x7f835c150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8361fc8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835c15f0c0 0x7f835c150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54406/0 (socket says 172.21.15.31:54406) 2024-01-19T04:57:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.223+0000 7f8361fc8640 1 -- 172.21.15.31:0/338371668 learned_addr learned my addr 172.21.15.31:0/338371668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f83617c7640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8361fc8640 1 -- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f835c15eb70 msgr2=0x7f835c0ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8360fc6640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f835c15eb70 0x7f835c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8361fc8640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f835c15eb70 0x7f835c0ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8361fc8640 1 -- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 msgr2=0x7f835c0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8361fc8640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c0c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8361fc8640 1 -- 172.21.15.31:0/338371668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350009400 con 0x7f835c15f0c0 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f83617c7640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8360fc6640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f835c15eb70 0x7f835c0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8361fc8640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835c15f0c0 0x7f835c150800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f835400ca70 tx=0x7f835400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f834dffb640 1 -- 172.21.15.31:0/338371668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835400abd0 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f834dffb640 1 -- 172.21.15.31:0/338371668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835400ad30 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f834dffb640 1 -- 172.21.15.31:0/338371668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835400aea0 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c0af330 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.224+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c14fa00 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.225+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c0af850 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.226+0000 7f834dffb640 1 -- 172.21.15.31:0/338371668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8354012640 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.226+0000 7f834dffb640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f834006af00 0x7f834006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.226+0000 7f834dffb640 1 -- 172.21.15.31:0/338371668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8354089b20 con 0x7f835c15f0c0 2024-01-19T04:57:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.226+0000 7f83617c7640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f834006af00 0x7f834006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.227+0000 7f83617c7640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f834006af00 0x7f834006d3c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8350036000 tx=0x7f8350008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.230+0000 7f834dffb640 1 -- 172.21.15.31:0/338371668 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f835c0af850 con 0x7f835c15f0c0 2024-01-19T04:57:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.445+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f835c0af850 con 0x7f835c15f0c0 2024-01-19T04:57:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.445+0000 7f834dffb640 1 -- 172.21.15.31:0/338371668 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f835c0af850 con 0x7f835c15f0c0 2024-01-19T04:57:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:40.445 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/3002045021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3002045021},{"type":"v1","addr":"172.21.15.137:6836","nonce":3002045021}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1415057458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":1415057458},{"type":"v1","addr":"172.21.15.137:6837","nonce":1415057458}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5919,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2062734261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2062734261},{"type":"v1","addr":"172.21.15.31:6833","nonce":2062734261}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:57:12.880027+0000","modified":"2024-01-19T04:57:15.888893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1600502506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1600502506},{"type":"v1","addr":"172.21.15.31:6835","nonce":1600502506}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-19T04:57:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f834006af00 msgr2=0x7f834006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f834006af00 0x7f834006d3c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8350036000 tx=0x7f8350008030 comp rx=0 tx=0).stop 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835c15f0c0 msgr2=0x7f835c150800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835c15f0c0 0x7f835c150800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f835400ca70 tx=0x7f835400cf30 comp rx=0 tx=0).stop 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 shutdown_connections 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f834006af00 0x7f834006d3c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835c15f0c0 0x7f835c150800 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f835c15eb70 0x7f835c0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 --2- 172.21.15.31:0/338371668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f835c15d8a0 0x7f835c0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 >> 172.21.15.31:0/338371668 conn(0x7f835c06abe0 msgr2=0x7f835c160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 shutdown_connections 2024-01-19T04:57:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.447+0000 7f8363a2b640 1 -- 172.21.15.31:0/338371668 wait complete. 2024-01-19T04:57:40.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:40.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.560+0000 7fd72699e640 1 Processor -- start 2024-01-19T04:57:40.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.561+0000 7fd72699e640 1 -- start start 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.561+0000 7fd72699e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd720161ee0 0x7fd7200b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.561+0000 7fd72699e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200d22b0 con 0x7fd7200d02f0 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.561+0000 7fd72699e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200d41c0 con 0x7fd7200cf150 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.561+0000 7fd72699e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7200d1980 con 0x7fd720161ee0 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.561+0000 7fd724f3b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd7200d02f0 0x7fd7200b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39490/0 (socket says 172.21.15.31:39490) 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.561+0000 7fd724f3b640 1 -- 172.21.15.31:0/4119332090 learned_addr learned my addr 172.21.15.31:0/4119332090 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523005167 0 0) 0x7fd7200d22b0 con 0x7fd7200d02f0 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd708003520 con 0x7fd7200d02f0 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71ffff640 1 --2- 172.21.15.31:0/4119332090 >> v2:172.21.15.137:3300/0 conn(0x7fd720161ee0 0x7fd7200b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3191916067 0 0) 0x7fd708003520 con 0x7fd7200d02f0 2024-01-19T04:57:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 >> v2:172.21.15.137:3300/0 conn(0x7fd720161ee0 msgr2=0x7fd7200b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 --2- 172.21.15.31:0/4119332090 >> v2:172.21.15.137:3300/0 conn(0x7fd720161ee0 0x7fd7200b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 >> v1:172.21.15.137:6789/0 conn(0x7fd7200cf150 legacy=0x7fd7200cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7200d1c60 con 0x7fd7200d02f0 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fd7180019f0 con 0x7fd7200d02f0 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 >> v1:172.21.15.31:6789/0 conn(0x7fd7200d02f0 legacy=0x7fd7200b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd708003970 0x7fd708003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7080047c0 0x7fd708004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd70800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200d1980 con 0x7fd708005180 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70800c020 con 0x7fd708003970 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70800c1e0 con 0x7fd7080047c0 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd724f3b640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd70800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71ffff640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd708003970 0x7fd708003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd71f7fe640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7080047c0 0x7fd708004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.562+0000 7fd724f3b640 1 -- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7080047c0 msgr2=0x7fd708004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd724f3b640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd7080047c0 0x7fd708004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd724f3b640 1 -- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd708003970 msgr2=0x7fd708003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd724f3b640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd708003970 0x7fd708003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd724f3b640 1 -- 172.21.15.31:0/4119332090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd708003520 con 0x7fd708005180 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd724f3b640 1 -- 172.21.15.31:0/4119332090 reap_dead start 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd724f3b640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd70800b980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd72008d3a0 tx=0x7fd7080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1881544543824761 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180072d0 con 0x7fd708005180 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71802f070 con 0x7fd708005180 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.563+0000 7fd71effd640 1 -- 172.21.15.31:0/4119332090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718007670 con 0x7fd708005180 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.564+0000 7fd72699e640 1 -- 172.21.15.31:0/4119332090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 msgr2=0x7fd70800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.564+0000 7fd72699e640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd70800b980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd72008d3a0 tx=0x7fd7080057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.564+0000 7fd72699e640 1 -- 172.21.15.31:0/4119332090 shutdown_connections 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.564+0000 7fd72699e640 1 --2- 172.21.15.31:0/4119332090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd70800b980 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.564+0000 7fd72699e640 1 -- 172.21.15.31:0/4119332090 >> 172.21.15.31:0/4119332090 conn(0x7fd72009c680 msgr2=0x7fd7200d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:40.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.564+0000 7fd72699e640 1 -- 172.21.15.31:0/4119332090 shutdown_connections 2024-01-19T04:57:40.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.564+0000 7fd72699e640 1 -- 172.21.15.31:0/4119332090 wait complete. 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.565+0000 7fd72699e640 1 Processor -- start 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.565+0000 7fd72699e640 1 -- start start 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd72699e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd720152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd72699e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd72006c480 0x7fd72006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd72699e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd720158a90 0x7fd720158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd72699e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72006ce40 con 0x7fd708005180 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd72699e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200ca470 con 0x7fd720158a90 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd72699e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200caff0 con 0x7fd72006c480 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd720152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd720152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54432/0 (socket says 172.21.15.31:54432) 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 -- 172.21.15.31:0/3824949843 learned_addr learned my addr 172.21.15.31:0/3824949843 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 -- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd72006c480 msgr2=0x7fd72006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd72006c480 0x7fd72006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 -- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd720158a90 msgr2=0x7fd720158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd720158a90 0x7fd720158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.566+0000 7fd71ffff640 1 -- 172.21.15.31:0/3824949843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd720152c50 con 0x7fd708005180 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.567+0000 7fd71ffff640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd720152710 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd714011910 tx=0x7fd714011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.567+0000 7fd71cff9640 1 -- 172.21.15.31:0/3824949843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400f9d0 con 0x7fd708005180 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.567+0000 7fd71cff9640 1 -- 172.21.15.31:0/3824949843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400fb30 con 0x7fd708005180 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.567+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7200ca6c0 con 0x7fd708005180 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.567+0000 7fd71cff9640 1 -- 172.21.15.31:0/3824949843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400fec0 con 0x7fd708005180 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.567+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7200cac80 con 0x7fd708005180 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.568+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd720074a80 con 0x7fd708005180 2024-01-19T04:57:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.568+0000 7fd71cff9640 1 -- 172.21.15.31:0/3824949843 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fd714014440 con 0x7fd708005180 2024-01-19T04:57:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.568+0000 7fd71cff9640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd70c06af00 0x7fd70c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.569+0000 7fd71cff9640 1 -- 172.21.15.31:0/3824949843 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd714089d60 con 0x7fd708005180 2024-01-19T04:57:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.569+0000 7fd71f7fe640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd70c06af00 0x7fd70c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.569+0000 7fd71f7fe640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd70c06af00 0x7fd70c06d3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd710000c00 tx=0x7fd710009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.573+0000 7fd71cff9640 1 -- 172.21.15.31:0/3824949843 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd714054cc0 con 0x7fd708005180 2024-01-19T04:57:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.751+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7200696e0 con 0x7fd708005180 2024-01-19T04:57:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.752+0000 7fd71cff9640 1 -- 172.21.15.31:0/3824949843 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fd7140582e0 con 0x7fd708005180 2024-01-19T04:57:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:40.753 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd70c06af00 msgr2=0x7fd70c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd70c06af00 0x7fd70c06d3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd710000c00 tx=0x7fd710009040 comp rx=0 tx=0).stop 2024-01-19T04:57:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 msgr2=0x7fd720152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd720152710 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd714011910 tx=0x7fd714011dd0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 shutdown_connections 2024-01-19T04:57:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fd70c06af00 0x7fd70c06d3c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd720158a90 0x7fd720158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd72006c480 0x7fd72006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 --2- 172.21.15.31:0/3824949843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd708005180 0x7fd720152710 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 >> 172.21.15.31:0/3824949843 conn(0x7fd72009c680 msgr2=0x7fd7200b8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 shutdown_connections 2024-01-19T04:57:40.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.755+0000 7fd72699e640 1 -- 172.21.15.31:0/3824949843 wait complete. 2024-01-19T04:57:40.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-19T04:55:05.602959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-19T04:57:10.830975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-01-19T04:57:11.850371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-19T04:57:40.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-19T04:57:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.871+0000 7f210b4af640 1 Processor -- start 2024-01-19T04:57:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.871+0000 7f210b4af640 1 -- start start 2024-01-19T04:57:40.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f210b4af640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2104143440 0x7f21040cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f210b4af640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 0x7f21040b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f210b4af640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210415ecb0 con 0x7f21040b33f0 2024-01-19T04:57:40.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f210b4af640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2104072440 con 0x7f2104143440 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f210b4af640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2104143820 con 0x7f210406ddc0 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f2108a4a640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f21040b33f0 0x7f210406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39504/0 (socket says 172.21.15.31:39504) 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f2108a4a640 1 -- 172.21.15.31:0/1472365434 learned_addr learned my addr 172.21.15.31:0/1472365434 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.872+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854116525 0 0) 0x7f210415ecb0 con 0x7f21040b33f0 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20f0003520 con 0x7f21040b33f0 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f210924b640 1 --2- 172.21.15.31:0/1472365434 >> v2:172.21.15.137:3300/0 conn(0x7f2104143440 0x7f21040cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2109a4c640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 0x7f21040b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077427860 0 0) 0x7f20f0003520 con 0x7f21040b33f0 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 msgr2=0x7f21040b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 0x7f21040b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 >> v2:172.21.15.137:3300/0 conn(0x7f2104143440 msgr2=0x7f21040cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 --2- 172.21.15.31:0/1472365434 >> v2:172.21.15.137:3300/0 conn(0x7f2104143440 0x7f21040cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210415e130 con 0x7f21040b33f0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f20f4001670 con 0x7f21040b33f0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 >> v1:172.21.15.31:6789/0 conn(0x7f21040b33f0 legacy=0x7f210406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f00039a0 0x7f20f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00047c0 0x7f20f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 0x7f20f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2104072440 con 0x7f20f00047c0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2104143820 con 0x7f20f00039a0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f000b9d0 con 0x7f210406ddc0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2108a4a640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00047c0 0x7f20f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f210924b640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f00039a0 0x7f20f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2109a4c640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 0x7f20f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2108a4a640 1 -- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 msgr2=0x7f20f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2108a4a640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f210406ddc0 0x7f20f0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2108a4a640 1 -- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f00039a0 msgr2=0x7f20f0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2108a4a640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f00039a0 0x7f20f0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.873+0000 7f2108a4a640 1 -- 172.21.15.31:0/1472365434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f0003520 con 0x7f20f00047c0 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.874+0000 7f2108a4a640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00047c0 0x7f20f0004c40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f21040b37f0 tx=0x7f20f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e54b0d3db0fb29 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.874+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4005ad0 con 0x7f20f00047c0 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.874+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f4005c50 con 0x7f20f00047c0 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.874+0000 7f20fbfff640 1 -- 172.21.15.31:0/1472365434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f402c850 con 0x7f20f00047c0 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.875+0000 7f210b4af640 1 -- 172.21.15.31:0/1472365434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00047c0 msgr2=0x7f20f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.875+0000 7f210b4af640 1 --2- 172.21.15.31:0/1472365434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f00047c0 0x7f20f0004c40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f21040b37f0 tx=0x7f20f00057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.875+0000 7f2109a4c640 1 -- 172.21.15.31:0/1472365434 reap_dead start 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.875+0000 7f210b4af640 1 -- 172.21.15.31:0/1472365434 shutdown_connections 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.875+0000 7f210b4af640 1 -- 172.21.15.31:0/1472365434 >> 172.21.15.31:0/1472365434 conn(0x7f210406aa00 msgr2=0x7f210406ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.875+0000 7f210b4af640 1 -- 172.21.15.31:0/1472365434 shutdown_connections 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.875+0000 7f210b4af640 1 -- 172.21.15.31:0/1472365434 wait complete. 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.876+0000 7f210b4af640 1 Processor -- start 2024-01-19T04:57:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.876+0000 7f210b4af640 1 -- start start 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210b4af640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20f00047c0 0x7f210413e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210b4af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2104159170 0x7f2104140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210b4af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21041ef050 0x7f21041ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210b4af640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210415bc50 con 0x7f2104159170 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210b4af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210415bef0 con 0x7f21041ef050 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210b4af640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210415c1d0 con 0x7f20f00047c0 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2104159170 0x7f2104140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2104159170 0x7f2104140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54456/0 (socket says 172.21.15.31:54456) 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 -- 172.21.15.31:0/1364058613 learned_addr learned my addr 172.21.15.31:0/1364058613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210924b640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20f00047c0 0x7f210413e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2109a4c640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21041ef050 0x7f21041ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 -- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20f00047c0 msgr2=0x7f210413e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20f00047c0 0x7f210413e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 -- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21041ef050 msgr2=0x7f21041ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21041ef050 0x7f21041ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 -- 172.21.15.31:0/1364058613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210415c4b0 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210924b640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20f00047c0 0x7f210413e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f2108a4a640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2104159170 0x7f2104140340 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c250 tx=0x7f21040b1e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f20f9ffb640 1 -- 172.21.15.31:0/1364058613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4039070 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f20f9ffb640 1 -- 172.21.15.31:0/1364058613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f402cc60 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210415c790 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.877+0000 7f20f9ffb640 1 -- 172.21.15.31:0/1364058613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4007070 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.878+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2104140880 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.879+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210415c950 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.879+0000 7f20f9ffb640 1 -- 172.21.15.31:0/1364058613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f20f4032070 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.879+0000 7f20f9ffb640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f20e806ae40 0x7f20e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.879+0000 7f20f9ffb640 1 -- 172.21.15.31:0/1364058613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20f403c040 con 0x7f2104159170 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.880+0000 7f210924b640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f20e806ae40 0x7f20e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.880+0000 7f210924b640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f20e806ae40 0x7f20e806d300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009bc0 tx=0x7f20fc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:40.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:40.884+0000 7f20f9ffb640 1 -- 172.21.15.31:0/1364058613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f20f40758f0 con 0x7f2104159170 2024-01-19T04:57:41.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.060+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2104074900 con 0x7f2104159170 2024-01-19T04:57:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.060+0000 7f20f9ffb640 1 -- 172.21.15.31:0/1364058613 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20f4078f10 con 0x7f2104159170 2024-01-19T04:57:41.060 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f20e806ae40 msgr2=0x7f20e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f20e806ae40 0x7f20e806d300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009bc0 tx=0x7f20fc008040 comp rx=0 tx=0).stop 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2104159170 msgr2=0x7f2104140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2104159170 0x7f2104140340 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c250 tx=0x7f21040b1e60 comp rx=0 tx=0).stop 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 shutdown_connections 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f20e806ae40 0x7f20e806d300 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21041ef050 0x7f21041ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2104159170 0x7f2104140340 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 --2- 172.21.15.31:0/1364058613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f20f00047c0 0x7f210413e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 >> 172.21.15.31:0/1364058613 conn(0x7f210406aa00 msgr2=0x7f2104070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 shutdown_connections 2024-01-19T04:57:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.062+0000 7f210b4af640 1 -- 172.21.15.31:0/1364058613 wait complete. 2024-01-19T04:57:41.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-19T04:57:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.176+0000 7fa9f9c43640 1 Processor -- start 2024-01-19T04:57:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.176+0000 7fa9f9c43640 1 -- start start 2024-01-19T04:57:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f9c43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f9c43640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9f40cd2e0 0x7fa9f40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f9c43640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f40cce50 con 0x7fa9f415b6b0 2024-01-19T04:57:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f9c43640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f40d03b0 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f9c43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40cc240 con 0x7fa9f40ce090 2024-01-19T04:57:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9f40cd2e0 0x7fa9f40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f37fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa9f415b6b0 0x7fa9f40cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39508/0 (socket says 172.21.15.31:39508) 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.177+0000 7fa9f37fe640 1 -- 172.21.15.31:0/4236003220 learned_addr learned my addr 172.21.15.31:0/4236003220 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/4236003220 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f3fff640 1 -- 172.21.15.31:0/4236003220 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 msgr2=0x7fa9f40d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/4236003220 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f3fff640 1 -- 172.21.15.31:0/4236003220 >> v1:172.21.15.31:6789/0 conn(0x7fa9f415b6b0 legacy=0x7fa9f40cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f3fff640 1 -- 172.21.15.31:0/4236003220 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40cc520 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f27fc640 1 -- 172.21.15.31:0/4236003220 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819699567 0 0) 0x7fa9f40cce50 con 0x7fa9f415b6b0 2024-01-19T04:57:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/4236003220 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/4236003220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f40b3f50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009d90 tx=0x7fa9ec02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b78c26c1c66cb3b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.178+0000 7fa9f27fc640 1 -- 172.21.15.31:0/4236003220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec02cce0 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f27fc640 1 -- 172.21.15.31:0/4236003220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec02ce40 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f27fc640 1 -- 172.21.15.31:0/4236003220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec034660 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 -- 172.21.15.31:0/4236003220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 msgr2=0x7fa9f40b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/4236003220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f40b3f50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009d90 tx=0x7fa9ec02bf30 comp rx=0 tx=0).stop 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 -- 172.21.15.31:0/4236003220 shutdown_connections 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/4236003220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f40b3f50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/4236003220 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 -- 172.21.15.31:0/4236003220 >> 172.21.15.31:0/4236003220 conn(0x7fa9f40d1100 msgr2=0x7fa9f40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 -- 172.21.15.31:0/4236003220 shutdown_connections 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.179+0000 7fa9f9c43640 1 -- 172.21.15.31:0/4236003220 wait complete. 2024-01-19T04:57:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.180+0000 7fa9f9c43640 1 Processor -- start 2024-01-19T04:57:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.180+0000 7fa9f9c43640 1 -- start start 2024-01-19T04:57:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f9c43640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f4153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f9c43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f9c43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9f415b6b0 0x7fa9f40b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f9c43640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4154490 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f9c43640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40c5530 con 0x7fa9f415b6b0 2024-01-19T04:57:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f9c43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40c5810 con 0x7fa9f40ce090 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f4153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f4153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54484/0 (socket says 172.21.15.31:54484) 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f37fe640 1 -- 172.21.15.31:0/704404006 learned_addr learned my addr 172.21.15.31:0/704404006 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9f415b6b0 0x7fa9f40b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.181+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f37fe640 1 -- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 msgr2=0x7fa9f40c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f37fe640 1 -- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9f415b6b0 msgr2=0x7fa9f40b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9f415b6b0 0x7fa9f40b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f37fe640 1 -- 172.21.15.31:0/704404006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec009a70 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9f415b6b0 0x7fa9f40b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f4153f50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4001eb0 tx=0x7fa9e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9dbfff640 1 -- 172.21.15.31:0/704404006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40074e0 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9dbfff640 1 -- 172.21.15.31:0/704404006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e4007640 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f40c5af0 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9dbfff640 1 -- 172.21.15.31:0/704404006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40059e0 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.182+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f40c60b0 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.183+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4074900 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.183+0000 7fa9dbfff640 1 -- 172.21.15.31:0/704404006 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa9e4005b40 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.184+0000 7fa9dbfff640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9e006aed0 0x7fa9e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.184+0000 7fa9dbfff640 1 -- 172.21.15.31:0/704404006 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9e4088c00 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.184+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9e006aed0 0x7fa9e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.184+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9e006aed0 0x7fa9e006d390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800a990 tx=0x7fa9e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.188+0000 7fa9dbfff640 1 -- 172.21.15.31:0/704404006 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa9e408f080 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.371+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9f4067030 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.372+0000 7fa9dbfff640 1 -- 172.21.15.31:0/704404006 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9e40170a0 con 0x7fa9f40cd2e0 2024-01-19T04:57:41.372 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9e006aed0 msgr2=0x7fa9e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9e006aed0 0x7fa9e006d390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800a990 tx=0x7fa9e8008040 comp rx=0 tx=0).stop 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 msgr2=0x7fa9f4153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f4153f50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4001eb0 tx=0x7fa9e400c890 comp rx=0 tx=0).stop 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 shutdown_connections 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9e006aed0 0x7fa9e006d390 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9f415b6b0 0x7fa9f40b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9f40ce090 0x7fa9f40c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 --2- 172.21.15.31:0/704404006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f40cd2e0 0x7fa9f4153f50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 >> 172.21.15.31:0/704404006 conn(0x7fa9f40d1100 msgr2=0x7fa9f406fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 shutdown_connections 2024-01-19T04:57:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.374+0000 7fa9f9c43640 1 -- 172.21.15.31:0/704404006 wait complete. 2024-01-19T04:57:41.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-19T04:57:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.489+0000 7fc0d5ca5640 1 Processor -- start 2024-01-19T04:57:41.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.489+0000 7fc0d5ca5640 1 -- start start 2024-01-19T04:57:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0d5ca5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc0d00cfa00 0x7fc0d0166080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0d5ca5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d015fc30 con 0x7fc0d015edd0 2024-01-19T04:57:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0d5ca5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d006c640 con 0x7fc0d013dfc0 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0d5ca5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d006c920 con 0x7fc0d00cfa00 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ceffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc0d015edd0 0x7fc0d00cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39512/0 (socket says 172.21.15.31:39512) 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ceffd640 1 -- 172.21.15.31:0/244665716 learned_addr learned my addr 172.21.15.31:0/244665716 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200022465 0 0) 0x7fc0d015fc30 con 0x7fc0d015edd0 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b8003520 con 0x7fc0d015edd0 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0cffff640 1 --2- 172.21.15.31:0/244665716 >> v2:172.21.15.137:3300/0 conn(0x7fc0d00cfa00 0x7fc0d0166080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970071236 0 0) 0x7fc0b8003520 con 0x7fc0d015edd0 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 >> v2:172.21.15.137:3300/0 conn(0x7fc0d00cfa00 msgr2=0x7fc0d0166080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 --2- 172.21.15.31:0/244665716 >> v2:172.21.15.137:3300/0 conn(0x7fc0d00cfa00 0x7fc0d0166080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 >> v1:172.21.15.137:6789/0 conn(0x7fc0d013dfc0 legacy=0x7fc0d015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d006cc00 con 0x7fc0d015edd0 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fc0c4001670 con 0x7fc0d015edd0 2024-01-19T04:57:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.490+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 >> v1:172.21.15.31:6789/0 conn(0x7fc0d015edd0 legacy=0x7fc0d00cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ce7fc640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0b8003970 0x7fc0b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ce7fc640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0b80047c0 0x7fc0b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ce7fc640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d006c920 con 0x7fc0b8005180 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b800c020 con 0x7fc0b80047c0 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b800c1e0 con 0x7fc0b8003970 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cffff640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cf7fe640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0b8003970 0x7fc0b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ceffd640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0b80047c0 0x7fc0b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cffff640 1 -- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0b8003970 msgr2=0x7fc0b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cffff640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0b8003970 0x7fc0b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cffff640 1 -- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0b80047c0 msgr2=0x7fc0b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cffff640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0b80047c0 0x7fc0b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cffff640 1 -- 172.21.15.31:0/244665716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b8003520 con 0x7fc0b8005180 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0cf7fe640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0b8003970 0x7fc0b8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.491+0000 7fc0ceffd640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0b80047c0 0x7fc0b8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0cffff640 1 -- 172.21.15.31:0/244665716 reap_dead start 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0cffff640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0b800b980 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009fc0 tx=0x7fc0b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e949434c6ca15b0b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c802cc60 con 0x7fc0b8005180 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c800f040 con 0x7fc0b8005180 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0ce7fc640 1 -- 172.21.15.31:0/244665716 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8003990 con 0x7fc0b8005180 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/244665716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 msgr2=0x7fc0b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0b800b980 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009fc0 tx=0x7fc0b80057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/244665716 shutdown_connections 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/244665716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0b800b980 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/244665716 >> 172.21.15.31:0/244665716 conn(0x7fc0d006aa00 msgr2=0x7fc0d006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/244665716 shutdown_connections 2024-01-19T04:57:41.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.492+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/244665716 wait complete. 2024-01-19T04:57:41.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.493+0000 7fc0d5ca5640 1 Processor -- start 2024-01-19T04:57:41.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.494+0000 7fc0d5ca5640 1 -- start start 2024-01-19T04:57:41.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.494+0000 7fc0d5ca5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0d0151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.494+0000 7fc0d5ca5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0d0158550 0x7fc0d01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.494+0000 7fc0d5ca5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d0153ab0 0x7fc0d0153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.494+0000 7fc0d5ca5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d0154800 con 0x7fc0b8005180 2024-01-19T04:57:41.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.494+0000 7fc0d5ca5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d0156840 con 0x7fc0d0153ab0 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.494+0000 7fc0d5ca5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d0158f10 con 0x7fc0d0158550 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0d0151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0d0151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54512/0 (socket says 172.21.15.31:54512) 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 -- 172.21.15.31:0/2937350898 learned_addr learned my addr 172.21.15.31:0/2937350898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cffff640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d0153ab0 0x7fc0d0153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0ceffd640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0d0158550 0x7fc0d01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 -- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0d0158550 msgr2=0x7fc0d01589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0d0158550 0x7fc0d01589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 -- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d0153ab0 msgr2=0x7fc0d0153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d0153ab0 0x7fc0d0153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 -- 172.21.15.31:0/2937350898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0155f10 con 0x7fc0b8005180 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cffff640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d0153ab0 0x7fc0d0153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0cf7fe640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0d0151010 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000e4b0 tx=0x7fc0c000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0b7fff640 1 -- 172.21.15.31:0/2937350898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00052e0 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0b7fff640 1 -- 172.21.15.31:0/2937350898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c0005440 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d01561f0 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0b7fff640 1 -- 172.21.15.31:0/2937350898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00037f0 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.495+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d0151550 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.496+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d0074900 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.497+0000 7fc0b7fff640 1 -- 172.21.15.31:0/2937350898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc0c0003950 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.497+0000 7fc0b7fff640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc0bc06aed0 0x7fc0bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.497+0000 7fc0b7fff640 1 -- 172.21.15.31:0/2937350898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0c0089250 con 0x7fc0b8005180 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.497+0000 7fc0ceffd640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc0bc06aed0 0x7fc0bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.498+0000 7fc0ceffd640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc0bc06aed0 0x7fc0bc06d390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc0d01f3da0 tx=0x7fc0c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.501+0000 7fc0b7fff640 1 -- 172.21.15.31:0/2937350898 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc0c00541b0 con 0x7fc0b8005180 2024-01-19T04:57:41.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.682+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0d006e0a0 con 0x7fc0b8005180 2024-01-19T04:57:41.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.683+0000 7fc0b7fff640 1 -- 172.21.15.31:0/2937350898 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0c00577d0 con 0x7fc0b8005180 2024-01-19T04:57:41.682 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-01-19T04:57:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc0bc06aed0 msgr2=0x7fc0bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc0bc06aed0 0x7fc0bc06d390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc0d01f3da0 tx=0x7fc0c4009040 comp rx=0 tx=0).stop 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 msgr2=0x7fc0d0151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0d0151010 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000e4b0 tx=0x7fc0c000e970 comp rx=0 tx=0).stop 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 shutdown_connections 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc0bc06aed0 0x7fc0bc06d390 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d0153ab0 0x7fc0d0153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc0d0158550 0x7fc0d01589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 --2- 172.21.15.31:0/2937350898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b8005180 0x7fc0d0151010 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 >> 172.21.15.31:0/2937350898 conn(0x7fc0d006aa00 msgr2=0x7fc0d0071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 shutdown_connections 2024-01-19T04:57:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.685+0000 7fc0d5ca5640 1 -- 172.21.15.31:0/2937350898 wait complete. 2024-01-19T04:57:41.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:57:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.797+0000 7fe36461d640 1 Processor -- start 2024-01-19T04:57:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.797+0000 7fe36461d640 1 -- start start 2024-01-19T04:57:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe36461d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 0x7fe35c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe36461d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe35c15eb70 0x7fe35c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe36461d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c06bf30 con 0x7fe35c0cca00 2024-01-19T04:57:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe36461d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c0cb9e0 con 0x7fe35c15eb70 2024-01-19T04:57:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe36461d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0cbcc0 con 0x7fe35c15d8a0 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe362bba640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe35c0cca00 0x7fe35c15f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39524/0 (socket says 172.21.15.31:39524) 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe362bba640 1 -- 172.21.15.31:0/871728847 learned_addr learned my addr 172.21.15.31:0/871728847 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3623b9640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 0x7fe35c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe361bb8640 1 --2- 172.21.15.31:0/871728847 >> v2:172.21.15.137:3300/0 conn(0x7fe35c15eb70 0x7fe35c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987636146 0 0) 0x7fe35c06bf30 con 0x7fe35c0cca00 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348003520 con 0x7fe35c0cca00 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529578639 0 0) 0x7fe348003520 con 0x7fe35c0cca00 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 msgr2=0x7fe35c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 0x7fe35c06b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 >> v2:172.21.15.137:3300/0 conn(0x7fe35c15eb70 msgr2=0x7fe35c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 --2- 172.21.15.31:0/871728847 >> v2:172.21.15.137:3300/0 conn(0x7fe35c15eb70 0x7fe35c0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0cbfa0 con 0x7fe35c0cca00 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7fe3540019f0 con 0x7fe35c0cca00 2024-01-19T04:57:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 >> v1:172.21.15.31:6789/0 conn(0x7fe35c0cca00 legacy=0x7fe35c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe348003970 0x7fe348003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3480047c0 0x7fe348004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 0x7fe348005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0cb9e0 con 0x7fe3480047c0 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0cbcc0 con 0x7fe348003970 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.798+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34800b980 con 0x7fe35c15d8a0 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe361bb8640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3480047c0 0x7fe348004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe3623b9640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe348003970 0x7fe348003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe362bba640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 0x7fe348005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe361bb8640 1 -- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 msgr2=0x7fe348005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe361bb8640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 0x7fe348005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe361bb8640 1 -- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe348003970 msgr2=0x7fe348003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe361bb8640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe348003970 0x7fe348003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe361bb8640 1 -- 172.21.15.31:0/871728847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe348003520 con 0x7fe3480047c0 2024-01-19T04:57:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.799+0000 7fe362bba640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c15d8a0 0x7fe348005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe361bb8640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3480047c0 0x7fe348004c40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009af0 tx=0x7fe3480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a2060cf1f2288b server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c02c710 con 0x7fe3480047c0 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c02c870 con 0x7fe3480047c0 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe3613b7640 1 -- 172.21.15.31:0/871728847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c02cc00 con 0x7fe3480047c0 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe36461d640 1 -- 172.21.15.31:0/871728847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3480047c0 msgr2=0x7fe348004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe36461d640 1 --2- 172.21.15.31:0/871728847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3480047c0 0x7fe348004c40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009af0 tx=0x7fe3480057f0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe362bba640 1 -- 172.21.15.31:0/871728847 reap_dead start 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe36461d640 1 -- 172.21.15.31:0/871728847 shutdown_connections 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe36461d640 1 -- 172.21.15.31:0/871728847 >> 172.21.15.31:0/871728847 conn(0x7fe35c06abe0 msgr2=0x7fe35c066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe36461d640 1 -- 172.21.15.31:0/871728847 shutdown_connections 2024-01-19T04:57:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.800+0000 7fe36461d640 1 -- 172.21.15.31:0/871728847 wait complete. 2024-01-19T04:57:41.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.801+0000 7fe36461d640 1 Processor -- start 2024-01-19T04:57:41.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe36461d640 1 -- start start 2024-01-19T04:57:41.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe36461d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3480047c0 0x7fe35c0b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe36461d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35c1f7b20 0x7fe35c0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe36461d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c1f6ea0 0x7fe35c1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe36461d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1f7890 con 0x7fe35c1f7b20 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe36461d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c8bf0 con 0x7fe3480047c0 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe36461d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0cad00 con 0x7fe35c1f6ea0 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe361bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35c1f7b20 0x7fe35c0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe361bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35c1f7b20 0x7fe35c0c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54526/0 (socket says 172.21.15.31:54526) 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe361bb8640 1 -- 172.21.15.31:0/3802136750 learned_addr learned my addr 172.21.15.31:0/3802136750 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe3623b9640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3480047c0 0x7fe35c0b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.802+0000 7fe362bba640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c1f6ea0 0x7fe35c1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe361bb8640 1 -- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c1f6ea0 msgr2=0x7fe35c1f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe361bb8640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c1f6ea0 0x7fe35c1f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe361bb8640 1 -- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3480047c0 msgr2=0x7fe35c0b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe361bb8640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3480047c0 0x7fe35c0b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe361bb8640 1 -- 172.21.15.31:0/3802136750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0cafe0 con 0x7fe35c1f7b20 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe362bba640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c1f6ea0 0x7fe35c1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe361bb8640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35c1f7b20 0x7fe35c0c8330 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe34c037040 tx=0x7fe34c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe352ffd640 1 -- 172.21.15.31:0/3802136750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003810 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe352ffd640 1 -- 172.21.15.31:0/3802136750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c02de90 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0cb2c0 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe352ffd640 1 -- 172.21.15.31:0/3802136750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003bb0 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.803+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0b6f00 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.804+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c0696e0 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.804+0000 7fe352ffd640 1 -- 172.21.15.31:0/3802136750 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fe34c003d10 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.805+0000 7fe352ffd640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe34006af00 0x7fe34006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.805+0000 7fe352ffd640 1 -- 172.21.15.31:0/3802136750 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe34c0ab4b0 con 0x7fe35c1f7b20 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.805+0000 7fe3623b9640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe34006af00 0x7fe34006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.806+0000 7fe3623b9640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe34006af00 0x7fe34006d3c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe358009ae0 tx=0x7fe358009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:41.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:41.809+0000 7fe352ffd640 1 -- 172.21.15.31:0/3802136750 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe34c0764a0 con 0x7fe35c1f7b20 2024-01-19T04:57:42.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.025+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe35c0b71e0 con 0x7fe35c1f7b20 2024-01-19T04:57:42.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.026+0000 7fe352ffd640 1 -- 172.21.15.31:0/3802136750 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fe34c079ac0 con 0x7fe35c1f7b20 2024-01-19T04:57:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/3002045021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3002045021},{"type":"v1","addr":"172.21.15.137:6836","nonce":3002045021}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1415057458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":1415057458},{"type":"v1","addr":"172.21.15.137:6837","nonce":1415057458}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5919,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2062734261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2062734261},{"type":"v1","addr":"172.21.15.31:6833","nonce":2062734261}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:57:12.880027+0000","modified":"2024-01-19T04:57:15.888893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1600502506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1600502506},{"type":"v1","addr":"172.21.15.31:6835","nonce":1600502506}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-19T04:57:42.026 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-01-19T04:57:42.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe34006af00 msgr2=0x7fe34006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe34006af00 0x7fe34006d3c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe358009ae0 tx=0x7fe358009190 comp rx=0 tx=0).stop 2024-01-19T04:57:42.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35c1f7b20 msgr2=0x7fe35c0c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35c1f7b20 0x7fe35c0c8330 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe34c037040 tx=0x7fe34c004730 comp rx=0 tx=0).stop 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 shutdown_connections 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fe34006af00 0x7fe34006d3c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe35c1f6ea0 0x7fe35c1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35c1f7b20 0x7fe35c0c8330 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 --2- 172.21.15.31:0/3802136750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3480047c0 0x7fe35c0b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 >> 172.21.15.31:0/3802136750 conn(0x7fe35c06abe0 msgr2=0x7fe35c0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 shutdown_connections 2024-01-19T04:57:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.028+0000 7fe36461d640 1 -- 172.21.15.31:0/3802136750 wait complete. 2024-01-19T04:57:42.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:42.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.139+0000 7f14c3561640 1 Processor -- start 2024-01-19T04:57:42.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.139+0000 7f14c3561640 1 -- start start 2024-01-19T04:57:42.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c3561640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f14bc160a10 0x7f14bc0b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c3561640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f14bc0d2e50 0x7f14bc0d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c3561640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0b4d00 0x7f14bc0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c3561640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14bc160df0 con 0x7f14bc0d2e50 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c3561640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14bc0b5690 con 0x7f14bc160a10 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c3561640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc0d1ae0 con 0x7f14bc0b4d00 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c0afc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f14bc0d2e50 0x7f14bc0d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c0afc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f14bc0d2e50 0x7f14bc0d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54532/0 (socket says 172.21.15.31:54532) 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.140+0000 7f14c0afc640 1 -- 172.21.15.31:0/1759456290 learned_addr learned my addr 172.21.15.31:0/1759456290 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c12fd640 1 --2- 172.21.15.31:0/1759456290 >> v2:172.21.15.137:3300/0 conn(0x7f14bc160a10 0x7f14bc0b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c1afe640 1 --2- 172.21.15.31:0/1759456290 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0b4d00 0x7f14bc0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c0afc640 1 -- 172.21.15.31:0/1759456290 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0b4d00 msgr2=0x7f14bc0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c0afc640 1 --2- 172.21.15.31:0/1759456290 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0b4d00 0x7f14bc0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c0afc640 1 -- 172.21.15.31:0/1759456290 >> v2:172.21.15.137:3300/0 conn(0x7f14bc160a10 msgr2=0x7f14bc0b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c0afc640 1 --2- 172.21.15.31:0/1759456290 >> v2:172.21.15.137:3300/0 conn(0x7f14bc160a10 0x7f14bc0b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c0afc640 1 -- 172.21.15.31:0/1759456290 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0d1db0 con 0x7f14bc0d2e50 2024-01-19T04:57:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.141+0000 7f14c0afc640 1 --2- 172.21.15.31:0/1759456290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0d2e50 0x7f14bc0d15a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0b59b0 tx=0x7f14b002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4044c696458171e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14bbfff640 1 -- 172.21.15.31:0/1759456290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0030860 con 0x7f14bc0d2e50 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14bbfff640 1 -- 172.21.15.31:0/1759456290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b00309c0 con 0x7f14bc0d2e50 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14bbfff640 1 -- 172.21.15.31:0/1759456290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0030d50 con 0x7f14bc0d2e50 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 -- 172.21.15.31:0/1759456290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0d2e50 msgr2=0x7f14bc0d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 --2- 172.21.15.31:0/1759456290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0d2e50 0x7f14bc0d15a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0b59b0 tx=0x7f14b002b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 -- 172.21.15.31:0/1759456290 shutdown_connections 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 --2- 172.21.15.31:0/1759456290 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0b4d00 0x7f14bc0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 --2- 172.21.15.31:0/1759456290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0d2e50 0x7f14bc0d15a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 --2- 172.21.15.31:0/1759456290 >> v2:172.21.15.137:3300/0 conn(0x7f14bc160a10 0x7f14bc0b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 -- 172.21.15.31:0/1759456290 >> 172.21.15.31:0/1759456290 conn(0x7f14bc072310 msgr2=0x7f14bc0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 -- 172.21.15.31:0/1759456290 shutdown_connections 2024-01-19T04:57:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.142+0000 7f14c3561640 1 -- 172.21.15.31:0/1759456290 wait complete. 2024-01-19T04:57:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.143+0000 7f14c3561640 1 Processor -- start 2024-01-19T04:57:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.143+0000 7f14c3561640 1 -- start start 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c3561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0b4d00 0x7f14bc1f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c3561640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0d2e50 0x7f14bc15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c3561640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14bc160a10 0x7f14bc0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c3561640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1f3000 con 0x7f14bc0b4d00 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c3561640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc0cd230 con 0x7f14bc160a10 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c3561640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1f7360 con 0x7f14bc0d2e50 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0b4d00 0x7f14bc1f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0b4d00 0x7f14bc1f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54542/0 (socket says 172.21.15.31:54542) 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 -- 172.21.15.31:0/2872204827 learned_addr learned my addr 172.21.15.31:0/2872204827 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 -- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0d2e50 msgr2=0x7f14bc15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0d2e50 0x7f14bc15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 -- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14bc160a10 msgr2=0x7f14bc0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14bc160a10 0x7f14bc0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.144+0000 7f14c12fd640 1 -- 172.21.15.31:0/2872204827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00094d0 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.145+0000 7f14c12fd640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0b4d00 0x7f14bc1f6450 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f14b400d5f0 tx=0x7f14b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.145+0000 7f14b9ffb640 1 -- 172.21.15.31:0/2872204827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40125f0 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.145+0000 7f14b9ffb640 1 -- 172.21.15.31:0/2872204827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b4012750 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.145+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc1f6ac0 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.145+0000 7f14b9ffb640 1 -- 172.21.15.31:0/2872204827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4012b00 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.145+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14bc1f7080 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.146+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14bc0696e0 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.146+0000 7f14b9ffb640 1 -- 172.21.15.31:0/2872204827 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f14b4012c60 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.147+0000 7f14b9ffb640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f14a806aed0 0x7f14a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.147+0000 7f14b9ffb640 1 -- 172.21.15.31:0/2872204827 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14b4089b50 con 0x7f14bc0b4d00 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.147+0000 7f14c0afc640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f14a806aed0 0x7f14a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.147+0000 7f14c0afc640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f14a806aed0 0x7f14a806d390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f14b0036000 tx=0x7f14b003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.151+0000 7f14b9ffb640 1 -- 172.21.15.31:0/2872204827 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f14b4054a10 con 0x7f14bc0b4d00 2024-01-19T04:57:42.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.329+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14bc0b2f10 con 0x7f14bc0b4d00 2024-01-19T04:57:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.330+0000 7f14b9ffb640 1 -- 172.21.15.31:0/2872204827 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f14b4058030 con 0x7f14bc0b4d00 2024-01-19T04:57:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:42.330 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f14a806aed0 msgr2=0x7f14a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f14a806aed0 0x7f14a806d390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f14b0036000 tx=0x7f14b003b040 comp rx=0 tx=0).stop 2024-01-19T04:57:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0b4d00 msgr2=0x7f14bc1f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0b4d00 0x7f14bc1f6450 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f14b400d5f0 tx=0x7f14b400dab0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 shutdown_connections 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f14a806aed0 0x7f14a806d390 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14bc160a10 0x7f14bc0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f14bc0d2e50 0x7f14bc15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 --2- 172.21.15.31:0/2872204827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14bc0b4d00 0x7f14bc1f6450 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 >> 172.21.15.31:0/2872204827 conn(0x7f14bc072310 msgr2=0x7f14bc0cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 shutdown_connections 2024-01-19T04:57:42.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.332+0000 7f14c3561640 1 -- 172.21.15.31:0/2872204827 wait complete. 2024-01-19T04:57:42.342 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-19T04:57:42.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:57:42.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.444+0000 7f3e792c9640 1 Processor -- start 2024-01-19T04:57:42.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.444+0000 7f3e792c9640 1 -- start start 2024-01-19T04:57:42.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e792c9640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3e740d08a0 0x7f3e7415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e792c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e7406ecb0 0x7f3e740cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e792c9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e740d0040 con 0x7f3e7406e8d0 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e792c9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7415c800 con 0x7f3e7406ecb0 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e792c9640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e740cd500 con 0x7f3e740d08a0 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e72ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e7406e8d0 0x7f3e740cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39540/0 (socket says 172.21.15.31:39540) 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e737fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e7406ecb0 0x7f3e740cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e72ffd640 1 -- 172.21.15.31:0/1638411559 learned_addr learned my addr 172.21.15.31:0/1638411559 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e727fc640 1 --2- 172.21.15.31:0/1638411559 >> v2:172.21.15.137:3300/0 conn(0x7f3e740d08a0 0x7f3e7415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e71ffb640 1 -- 172.21.15.31:0/1638411559 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405858466 0 0) 0x7f3e740d0040 con 0x7f3e7406e8d0 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.445+0000 7f3e71ffb640 1 -- 172.21.15.31:0/1638411559 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e5c003520 con 0x7f3e7406e8d0 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e737fe640 1 -- 172.21.15.31:0/1638411559 >> v2:172.21.15.137:3300/0 conn(0x7f3e740d08a0 msgr2=0x7f3e7415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e737fe640 1 --2- 172.21.15.31:0/1638411559 >> v2:172.21.15.137:3300/0 conn(0x7f3e740d08a0 0x7f3e7415c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e737fe640 1 -- 172.21.15.31:0/1638411559 >> v1:172.21.15.31:6789/0 conn(0x7f3e7406e8d0 legacy=0x7f3e740cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e737fe640 1 -- 172.21.15.31:0/1638411559 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740cb800 con 0x7f3e7406ecb0 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e727fc640 1 --2- 172.21.15.31:0/1638411559 >> v2:172.21.15.137:3300/0 conn(0x7f3e740d08a0 0x7f3e7415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e737fe640 1 --2- 172.21.15.31:0/1638411559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406ecb0 0x7f3e740cd9e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3e740c4c10 tx=0x7f3e6402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23562ec35100fcb server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e71ffb640 1 -- 172.21.15.31:0/1638411559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6402cbf0 con 0x7f3e7406ecb0 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e71ffb640 1 -- 172.21.15.31:0/1638411559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6402cd50 con 0x7f3e7406ecb0 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.446+0000 7f3e71ffb640 1 -- 172.21.15.31:0/1638411559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64034660 con 0x7f3e7406ecb0 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 -- 172.21.15.31:0/1638411559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406ecb0 msgr2=0x7f3e740cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 --2- 172.21.15.31:0/1638411559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406ecb0 0x7f3e740cd9e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3e740c4c10 tx=0x7f3e6402be40 comp rx=0 tx=0).stop 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 -- 172.21.15.31:0/1638411559 shutdown_connections 2024-01-19T04:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 --2- 172.21.15.31:0/1638411559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406ecb0 0x7f3e740cd9e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 --2- 172.21.15.31:0/1638411559 >> v2:172.21.15.137:3300/0 conn(0x7f3e740d08a0 0x7f3e7415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 -- 172.21.15.31:0/1638411559 >> 172.21.15.31:0/1638411559 conn(0x7f3e740b2c60 msgr2=0x7f3e740b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 -- 172.21.15.31:0/1638411559 shutdown_connections 2024-01-19T04:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.447+0000 7f3e792c9640 1 -- 172.21.15.31:0/1638411559 wait complete. 2024-01-19T04:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.448+0000 7f3e792c9640 1 Processor -- start 2024-01-19T04:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.448+0000 7f3e792c9640 1 -- start start 2024-01-19T04:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e792c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406e8d0 0x7f3e740b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e792c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e7406ecb0 0x7f3e741f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e792c9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e740d08a0 0x7f3e740b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e792c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740b5060 con 0x7f3e7406e8d0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e792c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740b43e0 con 0x7f3e7406ecb0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e792c9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740b46c0 con 0x7f3e740d08a0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406e8d0 0x7f3e740b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406e8d0 0x7f3e740b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54566/0 (socket says 172.21.15.31:54566) 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 -- 172.21.15.31:0/2517185183 learned_addr learned my addr 172.21.15.31:0/2517185183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 -- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e740d08a0 msgr2=0x7f3e740b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e740d08a0 0x7f3e740b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 -- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e7406ecb0 msgr2=0x7f3e741f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e7406ecb0 0x7f3e741f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.449+0000 7f3e72ffd640 1 -- 172.21.15.31:0/2517185183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e64009a70 con 0x7f3e7406e8d0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.450+0000 7f3e72ffd640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406e8d0 0x7f3e740b3d50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c430 tx=0x7f3e6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.450+0000 7f3e5b7fe640 1 -- 172.21.15.31:0/2517185183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c005ce0 con 0x7f3e7406e8d0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.450+0000 7f3e5b7fe640 1 -- 172.21.15.31:0/2517185183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c005e40 con 0x7f3e7406e8d0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.450+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e740b49a0 con 0x7f3e7406e8d0 2024-01-19T04:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.450+0000 7f3e5b7fe640 1 -- 172.21.15.31:0/2517185183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0072f0 con 0x7f3e7406e8d0 2024-01-19T04:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.450+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e740b68a0 con 0x7f3e7406e8d0 2024-01-19T04:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.451+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e74074a80 con 0x7f3e7406e8d0 2024-01-19T04:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.451+0000 7f3e5b7fe640 1 -- 172.21.15.31:0/2517185183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3e6c007450 con 0x7f3e7406e8d0 2024-01-19T04:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.452+0000 7f3e5b7fe640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e6006aed0 0x7f3e6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.452+0000 7f3e5b7fe640 1 -- 172.21.15.31:0/2517185183 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e6c088ce0 con 0x7f3e7406e8d0 2024-01-19T04:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.452+0000 7f3e727fc640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e6006aed0 0x7f3e6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.452+0000 7f3e727fc640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e6006aed0 0x7f3e6006d390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006f00 tx=0x7f3e68008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.456+0000 7f3e5b7fe640 1 -- 172.21.15.31:0/2517185183 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3e6c057630 con 0x7f3e7406e8d0 2024-01-19T04:57:42.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.675+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e740b6a60 con 0x7f3e7406e8d0 2024-01-19T04:57:42.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.675+0000 7f3e5b7fe640 1 -- 172.21.15.31:0/2517185183 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f3e6c017020 con 0x7f3e7406e8d0 2024-01-19T04:57:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:42.675 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/3002045021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3002045021},{"type":"v1","addr":"172.21.15.137:6836","nonce":3002045021}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1415057458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":1415057458},{"type":"v1","addr":"172.21.15.137:6837","nonce":1415057458}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5919,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2062734261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2062734261},{"type":"v1","addr":"172.21.15.31:6833","nonce":2062734261}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:57:12.880027+0000","modified":"2024-01-19T04:57:15.888893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1600502506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1600502506},{"type":"v1","addr":"172.21.15.31:6835","nonce":1600502506}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-19T04:57:42.675 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-01-19T04:57:42.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e6006aed0 msgr2=0x7f3e6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e6006aed0 0x7f3e6006d390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006f00 tx=0x7f3e68008040 comp rx=0 tx=0).stop 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406e8d0 msgr2=0x7f3e740b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406e8d0 0x7f3e740b3d50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c430 tx=0x7f3e6c00c8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 shutdown_connections 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3e6006aed0 0x7f3e6006d390 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3e740d08a0 0x7f3e740b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e7406ecb0 0x7f3e741f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 --2- 172.21.15.31:0/2517185183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406e8d0 0x7f3e740b3d50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 >> 172.21.15.31:0/2517185183 conn(0x7f3e740b2c60 msgr2=0x7f3e740b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 shutdown_connections 2024-01-19T04:57:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.677+0000 7f3e792c9640 1 -- 172.21.15.31:0/2517185183 wait complete. 2024-01-19T04:57:42.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:57:42.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.789+0000 7f0fd9b7c640 1 Processor -- start 2024-01-19T04:57:42.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.789+0000 7f0fd9b7c640 1 -- start start 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd9b7c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fd40cfd10 0x7f0fd40d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd9b7c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40d0ef0 0x7f0fd40d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd9b7c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40d06b0 con 0x7f0fd4160bb0 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd9b7c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40d18c0 con 0x7f0fd40cfd10 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd9b7c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40d1b90 con 0x7f0fd40d0ef0 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd2ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fd40cfd10 0x7f0fd40d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd2ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fd40cfd10 0x7f0fd40d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54572/0 (socket says 172.21.15.31:54572) 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd2ffd640 1 -- 172.21.15.31:0/3219133808 learned_addr learned my addr 172.21.15.31:0/3219133808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd3fff640 1 --2- 172.21.15.31:0/3219133808 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40d0ef0 0x7f0fd40d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd2ffd640 1 -- 172.21.15.31:0/3219133808 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40d0ef0 msgr2=0x7f0fd40d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd2ffd640 1 --2- 172.21.15.31:0/3219133808 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40d0ef0 0x7f0fd40d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd2ffd640 1 -- 172.21.15.31:0/3219133808 >> v1:172.21.15.137:6789/0 conn(0x7f0fd4160bb0 legacy=0x7f0fd40b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.790+0000 7f0fd2ffd640 1 -- 172.21.15.31:0/3219133808 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40d1e70 con 0x7f0fd40cfd10 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd2ffd640 1 --2- 172.21.15.31:0/3219133808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd40cfd10 0x7f0fd40d0170 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009870 tx=0x7f0fc802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8057623fd42c07 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd27fc640 1 -- 172.21.15.31:0/3219133808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8030840 con 0x7f0fd40cfd10 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd27fc640 1 -- 172.21.15.31:0/3219133808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc80309a0 con 0x7f0fd40cfd10 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd27fc640 1 -- 172.21.15.31:0/3219133808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8030d30 con 0x7f0fd40cfd10 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/3219133808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd40cfd10 msgr2=0x7f0fd40d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/3219133808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd40cfd10 0x7f0fd40d0170 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009870 tx=0x7f0fc802b980 comp rx=0 tx=0).stop 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/3219133808 shutdown_connections 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/3219133808 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40d0ef0 0x7f0fd40d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/3219133808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd40cfd10 0x7f0fd40d0170 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.791+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/3219133808 >> 172.21.15.31:0/3219133808 conn(0x7f0fd409c600 msgr2=0x7f0fd40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.792+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/3219133808 shutdown_connections 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.792+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/3219133808 wait complete. 2024-01-19T04:57:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.793+0000 7f0fd9b7c640 1 Processor -- start 2024-01-19T04:57:42.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.793+0000 7f0fd9b7c640 1 -- start start 2024-01-19T04:57:42.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd9b7c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40cfd10 0x7f0fd415fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd9b7c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fd40d0ef0 0x7f0fd40cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd9b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd4160bb0 0x7f0fd40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd9b7c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40cd240 con 0x7f0fd4160bb0 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd9b7c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd41f2ee0 con 0x7f0fd40d0ef0 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd9b7c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd41f64d0 con 0x7f0fd40cfd10 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd4160bb0 0x7f0fd40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd4160bb0 0x7f0fd40ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54588/0 (socket says 172.21.15.31:54588) 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 -- 172.21.15.31:0/2188083358 learned_addr learned my addr 172.21.15.31:0/2188083358 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fd40d0ef0 0x7f0fd40cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd37fe640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40cfd10 0x7f0fd415fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 -- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40cfd10 msgr2=0x7f0fd415fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40cfd10 0x7f0fd415fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 -- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fd40d0ef0 msgr2=0x7f0fd40cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fd40d0ef0 0x7f0fd40cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 -- 172.21.15.31:0/2188083358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc80094d0 con 0x7f0fd4160bb0 2024-01-19T04:57:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fd3fff640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd4160bb0 0x7f0fd40ccd00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00efc0 tx=0x7f0fcc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.794+0000 7f0fbbfff640 1 -- 172.21.15.31:0/2188083358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc014480 con 0x7f0fd4160bb0 2024-01-19T04:57:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.795+0000 7f0fbbfff640 1 -- 172.21.15.31:0/2188083358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc00f040 con 0x7f0fd4160bb0 2024-01-19T04:57:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.795+0000 7f0fbbfff640 1 -- 172.21.15.31:0/2188083358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc014820 con 0x7f0fd4160bb0 2024-01-19T04:57:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.795+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd41f67b0 con 0x7f0fd4160bb0 2024-01-19T04:57:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.795+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd41f6d70 con 0x7f0fd4160bb0 2024-01-19T04:57:42.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.795+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd40696e0 con 0x7f0fd4160bb0 2024-01-19T04:57:42.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.796+0000 7f0fbbfff640 1 -- 172.21.15.31:0/2188083358 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0fcc014980 con 0x7f0fd4160bb0 2024-01-19T04:57:42.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.796+0000 7f0fbbfff640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0fc006aed0 0x7f0fc006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:42.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.796+0000 7f0fbbfff640 1 -- 172.21.15.31:0/2188083358 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fcc08a080 con 0x7f0fd4160bb0 2024-01-19T04:57:42.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.797+0000 7f0fd37fe640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0fc006aed0 0x7f0fc006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:42.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.797+0000 7f0fd37fe640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0fc006aed0 0x7f0fc006d390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40016e0 tx=0x7f0fc4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:42.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:42.801+0000 7f0fbbfff640 1 -- 172.21.15.31:0/2188083358 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0fcc055000 con 0x7f0fd4160bb0 2024-01-19T04:57:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.014+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fd408d3b0 con 0x7f0fd4160bb0 2024-01-19T04:57:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.015+0000 7f0fbbfff640 1 -- 172.21.15.31:0/2188083358 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f0fcc058620 con 0x7f0fd4160bb0 2024-01-19T04:57:43.015 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:43.015 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/3002045021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3002045021},{"type":"v1","addr":"172.21.15.137:6836","nonce":3002045021}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1415057458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":1415057458},{"type":"v1","addr":"172.21.15.137:6837","nonce":1415057458}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5919,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2062734261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2062734261},{"type":"v1","addr":"172.21.15.31:6833","nonce":2062734261}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-19T04:57:12.880027+0000","modified":"2024-01-19T04:57:15.888893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1600502506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1600502506},{"type":"v1","addr":"172.21.15.31:6835","nonce":1600502506}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-19T04:57:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-01-19T04:57:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0fc006aed0 msgr2=0x7f0fc006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0fc006aed0 0x7f0fc006d390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40016e0 tx=0x7f0fc4009040 comp rx=0 tx=0).stop 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd4160bb0 msgr2=0x7f0fd40ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd4160bb0 0x7f0fd40ccd00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00efc0 tx=0x7f0fcc00c490 comp rx=0 tx=0).stop 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 shutdown_connections 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0fc006aed0 0x7f0fc006d390 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fd4160bb0 0x7f0fd40ccd00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fd40d0ef0 0x7f0fd40cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 --2- 172.21.15.31:0/2188083358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0fd40cfd10 0x7f0fd415fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 >> 172.21.15.31:0/2188083358 conn(0x7f0fd409c600 msgr2=0x7f0fd40b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 shutdown_connections 2024-01-19T04:57:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.017+0000 7f0fd9b7c640 1 -- 172.21.15.31:0/2188083358 wait complete. 2024-01-19T04:57:43.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:43.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.129+0000 7fb235b05640 1 Processor -- start 2024-01-19T04:57:43.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.129+0000 7fb235b05640 1 -- start start 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb235b05640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb2301580f0 0x7fb23015f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb235b05640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb23015de70 0x7fb23015e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb235b05640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300cb8b0 con 0x7fb2300b2d40 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb235b05640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23006c300 con 0x7fb23015de70 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb235b05640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23015e960 con 0x7fb2301580f0 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb23015de70 0x7fb23015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb23015de70 0x7fb23015e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54598/0 (socket says 172.21.15.31:54598) 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22effd640 1 -- 172.21.15.31:0/3792483696 learned_addr learned my addr 172.21.15.31:0/3792483696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22f7fe640 1 --2- 172.21.15.31:0/3792483696 >> v2:172.21.15.137:3300/0 conn(0x7fb2301580f0 0x7fb23015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22effd640 1 -- 172.21.15.31:0/3792483696 >> v2:172.21.15.137:3300/0 conn(0x7fb2301580f0 msgr2=0x7fb23015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22effd640 1 --2- 172.21.15.31:0/3792483696 >> v2:172.21.15.137:3300/0 conn(0x7fb2301580f0 0x7fb23015f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22effd640 1 -- 172.21.15.31:0/3792483696 >> v1:172.21.15.137:6789/0 conn(0x7fb2300b2d40 legacy=0x7fb2300cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.130+0000 7fb22effd640 1 -- 172.21.15.31:0/3792483696 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23015ec40 con 0x7fb23015de70 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb22effd640 1 --2- 172.21.15.31:0/3792483696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23015de70 0x7fb23015e320 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb2300d2300 tx=0x7fb21802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6497af48b28c73e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb22e7fc640 1 -- 172.21.15.31:0/3792483696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218030860 con 0x7fb23015de70 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb22e7fc640 1 -- 172.21.15.31:0/3792483696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180309c0 con 0x7fb23015de70 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb22e7fc640 1 -- 172.21.15.31:0/3792483696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218030d50 con 0x7fb23015de70 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb235b05640 1 -- 172.21.15.31:0/3792483696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23015de70 msgr2=0x7fb23015e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb235b05640 1 --2- 172.21.15.31:0/3792483696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23015de70 0x7fb23015e320 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb2300d2300 tx=0x7fb21802b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb235b05640 1 -- 172.21.15.31:0/3792483696 shutdown_connections 2024-01-19T04:57:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb235b05640 1 --2- 172.21.15.31:0/3792483696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23015de70 0x7fb23015e320 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb235b05640 1 --2- 172.21.15.31:0/3792483696 >> v2:172.21.15.137:3300/0 conn(0x7fb2301580f0 0x7fb23015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb235b05640 1 -- 172.21.15.31:0/3792483696 >> 172.21.15.31:0/3792483696 conn(0x7fb23006abe0 msgr2=0x7fb230066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.131+0000 7fb235b05640 1 -- 172.21.15.31:0/3792483696 shutdown_connections 2024-01-19T04:57:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.132+0000 7fb235b05640 1 -- 172.21.15.31:0/3792483696 wait complete. 2024-01-19T04:57:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 Processor -- start 2024-01-19T04:57:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 -- start start 2024-01-19T04:57:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2300b2d40 0x7fb2300c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2301580f0 0x7fb2300c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb23015de70 0x7fb2300d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300d1460 con 0x7fb2300b2d40 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300cab00 con 0x7fb2301580f0 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.133+0000 7fb235b05640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300cade0 con 0x7fb23015de70 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2300b2d40 0x7fb2300c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2300b2d40 0x7fb2300c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54614/0 (socket says 172.21.15.31:54614) 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 -- 172.21.15.31:0/2079250750 learned_addr learned my addr 172.21.15.31:0/2079250750 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22effd640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2301580f0 0x7fb2300c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22ffff640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb23015de70 0x7fb2300d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 -- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb23015de70 msgr2=0x7fb2300d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb23015de70 0x7fb2300d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 -- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2301580f0 msgr2=0x7fb2300c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2301580f0 0x7fb2300c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 -- 172.21.15.31:0/2079250750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180094d0 con 0x7fb2300b2d40 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22effd640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2301580f0 0x7fb2300c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb22f7fe640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2300b2d40 0x7fb2300c8330 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e9b0 tx=0x7fb22000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb227fff640 1 -- 172.21.15.31:0/2079250750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220009670 con 0x7fb2300b2d40 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb227fff640 1 -- 172.21.15.31:0/2079250750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22000c9a0 con 0x7fb2300b2d40 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2301f8200 con 0x7fb2300b2d40 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb227fff640 1 -- 172.21.15.31:0/2079250750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000cd50 con 0x7fb2300b2d40 2024-01-19T04:57:43.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.134+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300d1e30 con 0x7fb2300b2d40 2024-01-19T04:57:43.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.135+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300d2080 con 0x7fb2300b2d40 2024-01-19T04:57:43.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.136+0000 7fb227fff640 1 -- 172.21.15.31:0/2079250750 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fb220013470 con 0x7fb2300b2d40 2024-01-19T04:57:43.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.136+0000 7fb227fff640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb21006af00 0x7fb21006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.136+0000 7fb227fff640 1 -- 172.21.15.31:0/2079250750 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb220089770 con 0x7fb2300b2d40 2024-01-19T04:57:43.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.136+0000 7fb22effd640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb21006af00 0x7fb21006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.137+0000 7fb22effd640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb21006af00 0x7fb21006d3c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb218036000 tx=0x7fb21803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.140+0000 7fb227fff640 1 -- 172.21.15.31:0/2079250750 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb2200546f0 con 0x7fb2300b2d40 2024-01-19T04:57:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.322+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2300696e0 con 0x7fb2300b2d40 2024-01-19T04:57:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.323+0000 7fb227fff640 1 -- 172.21.15.31:0/2079250750 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fb220057d10 con 0x7fb2300b2d40 2024-01-19T04:57:43.323 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:43.323 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:14.524879+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-01-19T04:57:10.830975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-01-19T04:57:11.850371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb21006af00 msgr2=0x7fb21006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb21006af00 0x7fb21006d3c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb218036000 tx=0x7fb21803b040 comp rx=0 tx=0).stop 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2300b2d40 msgr2=0x7fb2300c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2300b2d40 0x7fb2300c8330 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e9b0 tx=0x7fb22000ee70 comp rx=0 tx=0).stop 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 shutdown_connections 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fb21006af00 0x7fb21006d3c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb23015de70 0x7fb2300d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2301580f0 0x7fb2300c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 --2- 172.21.15.31:0/2079250750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2300b2d40 0x7fb2300c8330 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 >> 172.21.15.31:0/2079250750 conn(0x7fb23006abe0 msgr2=0x7fb2301602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 shutdown_connections 2024-01-19T04:57:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.325+0000 7fb235b05640 1 -- 172.21.15.31:0/2079250750 wait complete. 2024-01-19T04:57:43.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-19T04:57:43.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.439+0000 7f6bc4497640 1 Processor -- start 2024-01-19T04:57:43.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.439+0000 7f6bc4497640 1 -- start start 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc4497640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6bbc159a10 0x7f6bbc0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc4497640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc4497640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc06cd90 con 0x7f6bbc159630 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc4497640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc0b3f50 con 0x7f6bbc159a10 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc4497640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc070990 con 0x7f6bbc0b26e0 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc1a32640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6bbc159a10 0x7f6bbc0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc1a32640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6bbc159a10 0x7f6bbc0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54616/0 (socket says 172.21.15.31:54616) 2024-01-19T04:57:43.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc1a32640 1 -- 172.21.15.31:0/564382172 learned_addr learned my addr 172.21.15.31:0/564382172 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:43.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc2a34640 1 --2- 172.21.15.31:0/564382172 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc1a32640 1 -- 172.21.15.31:0/564382172 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 msgr2=0x7f6bbc070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:43.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc1a32640 1 --2- 172.21.15.31:0/564382172 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc1a32640 1 -- 172.21.15.31:0/564382172 >> v1:172.21.15.137:6789/0 conn(0x7f6bbc159630 legacy=0x7f6bbc0cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc1a32640 1 -- 172.21.15.31:0/564382172 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0b3c70 con 0x7f6bbc159a10 2024-01-19T04:57:43.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.440+0000 7f6bc2a34640 1 --2- 172.21.15.31:0/564382172 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:43.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.441+0000 7f6bc1a32640 1 --2- 172.21.15.31:0/564382172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159a10 0x7f6bbc0b35d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009870 tx=0x7f6bac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ca1603d5b0dec2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.441+0000 7f6bc1231640 1 -- 172.21.15.31:0/564382172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac030840 con 0x7f6bbc159a10 2024-01-19T04:57:43.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.441+0000 7f6bc1231640 1 -- 172.21.15.31:0/564382172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac0309a0 con 0x7f6bbc159a10 2024-01-19T04:57:43.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.441+0000 7f6bc1231640 1 -- 172.21.15.31:0/564382172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac030d30 con 0x7f6bbc159a10 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 -- 172.21.15.31:0/564382172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159a10 msgr2=0x7f6bbc0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 --2- 172.21.15.31:0/564382172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159a10 0x7f6bbc0b35d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009870 tx=0x7f6bac02b980 comp rx=0 tx=0).stop 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 -- 172.21.15.31:0/564382172 shutdown_connections 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 --2- 172.21.15.31:0/564382172 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 --2- 172.21.15.31:0/564382172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159a10 0x7f6bbc0b35d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 -- 172.21.15.31:0/564382172 >> 172.21.15.31:0/564382172 conn(0x7f6bbc06aa00 msgr2=0x7f6bbc06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 -- 172.21.15.31:0/564382172 shutdown_connections 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.442+0000 7f6bc4497640 1 -- 172.21.15.31:0/564382172 wait complete. 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.443+0000 7f6bc4497640 1 Processor -- start 2024-01-19T04:57:43.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.443+0000 7f6bc4497640 1 -- start start 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc4497640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc4497640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159630 0x7f6bbc15db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc4497640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bbc159a10 0x7f6bbc0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc4497640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc147b40 con 0x7f6bbc159630 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc4497640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc14c050 con 0x7f6bbc159a10 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc4497640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc14b7b0 con 0x7f6bbc0b26e0 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159630 0x7f6bbc15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159630 0x7f6bbc15db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54628/0 (socket says 172.21.15.31:54628) 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 -- 172.21.15.31:0/2442045509 learned_addr learned my addr 172.21.15.31:0/2442045509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc2233640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc2a34640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bbc159a10 0x7f6bbc0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 -- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 msgr2=0x7f6bbc14a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc14a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 -- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bbc159a10 msgr2=0x7f6bbc0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bbc159a10 0x7f6bbc0afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.444+0000 7f6bc1a32640 1 -- 172.21.15.31:0/2442045509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0094d0 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.445+0000 7f6bc1a32640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159630 0x7f6bbc15db80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02b9c0 tx=0x7f6bac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.445+0000 7f6bb2ffd640 1 -- 172.21.15.31:0/2442045509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac03bca0 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.445+0000 7f6bb2ffd640 1 -- 172.21.15.31:0/2442045509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac03be00 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.445+0000 7f6bb2ffd640 1 -- 172.21.15.31:0/2442045509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac045650 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.445+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc14ba90 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.445+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc145880 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.446+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc145a10 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.446+0000 7f6bb2ffd640 1 -- 172.21.15.31:0/2442045509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6bac02f390 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.446+0000 7f6bb2ffd640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ba006aed0 0x7f6ba006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.447+0000 7f6bb2ffd640 1 -- 172.21.15.31:0/2442045509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bac0ad950 con 0x7f6bbc159630 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.447+0000 7f6bc2233640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ba006aed0 0x7f6ba006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.447+0000 7f6bc2233640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ba006aed0 0x7f6ba006d390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8001780 tx=0x7f6bb8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.451+0000 7f6bb2ffd640 1 -- 172.21.15.31:0/2442045509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6bac0788c0 con 0x7f6bbc159630 2024-01-19T04:57:43.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.672+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6bbc147910 con 0x7f6bbc159630 2024-01-19T04:57:43.672 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-19T04:57:43.672+0000 7f70c6e70640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-19T04:57:43.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.674+0000 7f6bb2ffd640 1 -- 172.21.15.31:0/2442045509 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6bbc147910 con 0x7f6bbc159630 2024-01-19T04:57:43.673 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-19T04:57:43.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.676+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ba006aed0 msgr2=0x7f6ba006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.676+0000 7f6bc4497640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ba006aed0 0x7f6ba006d390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8001780 tx=0x7f6bb8009040 comp rx=0 tx=0).stop 2024-01-19T04:57:43.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.676+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159630 msgr2=0x7f6bbc15db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.676+0000 7f6bc4497640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159630 0x7f6bbc15db80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02b9c0 tx=0x7f6bac004730 comp rx=0 tx=0).stop 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 shutdown_connections 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6ba006aed0 0x7f6ba006d390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bbc159a10 0x7f6bbc0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc159630 0x7f6bbc15db80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 --2- 172.21.15.31:0/2442045509 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bbc0b26e0 0x7f6bbc14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 >> 172.21.15.31:0/2442045509 conn(0x7f6bbc06aa00 msgr2=0x7f6bbc0aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 shutdown_connections 2024-01-19T04:57:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.677+0000 7f6bc4497640 1 -- 172.21.15.31:0/2442045509 wait complete. 2024-01-19T04:57:43.677 INFO:tasks.ceph.mds.a.smithi031.stderr: -2605> 2024-01-19T04:57:18.880+0000 7f6d66227640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-19T04:57:43.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-19T04:57:43.704 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-01-19T04:57:43.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.792+0000 7f83d4dde640 1 Processor -- start 2024-01-19T04:57:43.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.792+0000 7f83d4dde640 1 -- start start 2024-01-19T04:57:43.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.792+0000 7f83d4dde640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f83d0161860 0x7f83d0168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.792+0000 7f83d4dde640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00cf9f0 con 0x7f83d00cf570 2024-01-19T04:57:43.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.792+0000 7f83d4dde640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00cfcd0 con 0x7f83d015d3a0 2024-01-19T04:57:43.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.792+0000 7f83d4dde640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d00cffb0 con 0x7f83d0161860 2024-01-19T04:57:43.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.793+0000 7f83ced9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f83d0161860 0x7f83d0168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.793+0000 7f83ced9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f83d0161860 0x7f83d0168490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54650/0 (socket says 172.21.15.31:54650) 2024-01-19T04:57:43.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.793+0000 7f83ced9d640 1 -- 172.21.15.31:0/3550707014 learned_addr learned my addr 172.21.15.31:0/3550707014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:43.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.793+0000 7f83ced9d640 1 -- 172.21.15.31:0/3550707014 >> v1:172.21.15.137:6789/0 conn(0x7f83d015d3a0 legacy=0x7f83d006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:43.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.793+0000 7f83ced9d640 1 -- 172.21.15.31:0/3550707014 >> v1:172.21.15.31:6789/0 conn(0x7f83d00cf570 legacy=0x7f83d0161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.793+0000 7f83ced9d640 1 -- 172.21.15.31:0/3550707014 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00d0290 con 0x7f83d0161860 2024-01-19T04:57:43.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.793+0000 7f83cdd9b640 1 -- 172.21.15.31:0/3550707014 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562911822 0 0) 0x7f83d00cf9f0 con 0x7f83d00cf570 2024-01-19T04:57:43.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83ced9d640 1 --2- 172.21.15.31:0/3550707014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0161860 0x7f83d0168490 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009e50 tx=0x7f83c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36022d48b9f941c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83cdd9b640 1 -- 172.21.15.31:0/3550707014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c802ca80 con 0x7f83d0161860 2024-01-19T04:57:43.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83cdd9b640 1 -- 172.21.15.31:0/3550707014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c802cbe0 con 0x7f83d0161860 2024-01-19T04:57:43.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83cdd9b640 1 -- 172.21.15.31:0/3550707014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c80343f0 con 0x7f83d0161860 2024-01-19T04:57:43.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83d4dde640 1 -- 172.21.15.31:0/3550707014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0161860 msgr2=0x7f83d0168490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83d4dde640 1 --2- 172.21.15.31:0/3550707014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0161860 0x7f83d0168490 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009e50 tx=0x7f83c802bee0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83d4dde640 1 -- 172.21.15.31:0/3550707014 shutdown_connections 2024-01-19T04:57:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83d4dde640 1 --2- 172.21.15.31:0/3550707014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d0161860 0x7f83d0168490 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83d4dde640 1 -- 172.21.15.31:0/3550707014 >> 172.21.15.31:0/3550707014 conn(0x7f83d00d1100 msgr2=0x7f83d00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83d4dde640 1 -- 172.21.15.31:0/3550707014 shutdown_connections 2024-01-19T04:57:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.794+0000 7f83d4dde640 1 -- 172.21.15.31:0/3550707014 wait complete. 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 Processor -- start 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 -- start start 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d00cf570 0x7f83d014b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83d015d3a0 0x7f83d00b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d0161860 0x7f83d00b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01e79d0 con 0x7f83d00cf570 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01e7cb0 con 0x7f83d0161860 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83d4dde640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00b0800 con 0x7f83d015d3a0 2024-01-19T04:57:43.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83ce59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d00cf570 0x7f83d014b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83ce59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d00cf570 0x7f83d014b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54662/0 (socket says 172.21.15.31:54662) 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.796+0000 7f83ce59c640 1 -- 172.21.15.31:0/2767659274 learned_addr learned my addr 172.21.15.31:0/2767659274 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83c7fff640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83d015d3a0 0x7f83d00b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ced9d640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d0161860 0x7f83d00b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ce59c640 1 -- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83d015d3a0 msgr2=0x7f83d00b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ce59c640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83d015d3a0 0x7f83d00b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ce59c640 1 -- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d0161860 msgr2=0x7f83d00b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ce59c640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d0161860 0x7f83d00b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ce59c640 1 -- 172.21.15.31:0/2767659274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8009b30 con 0x7f83d00cf570 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83c7fff640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83d015d3a0 0x7f83d00b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ced9d640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d0161860 0x7f83d00b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83ce59c640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d00cf570 0x7f83d014b4b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83c0004640 tx=0x7f83c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83c6ffd640 1 -- 172.21.15.31:0/2767659274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0011790 con 0x7f83d00cf570 2024-01-19T04:57:43.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83c6ffd640 1 -- 172.21.15.31:0/2767659274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c00118f0 con 0x7f83d00cf570 2024-01-19T04:57:43.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83c6ffd640 1 -- 172.21.15.31:0/2767659274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0011ce0 con 0x7f83d00cf570 2024-01-19T04:57:43.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d00b0ae0 con 0x7f83d00cf570 2024-01-19T04:57:43.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.797+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d014b9f0 con 0x7f83d00cf570 2024-01-19T04:57:43.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.798+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d00b0640 con 0x7f83d00cf570 2024-01-19T04:57:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.799+0000 7f83c6ffd640 1 -- 172.21.15.31:0/2767659274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f83c0012420 con 0x7f83d00cf570 2024-01-19T04:57:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.799+0000 7f83c6ffd640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83b006aed0 0x7f83b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.799+0000 7f83c6ffd640 1 -- 172.21.15.31:0/2767659274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f83c00892e0 con 0x7f83d00cf570 2024-01-19T04:57:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.799+0000 7f83c7fff640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83b006aed0 0x7f83b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.800+0000 7f83c7fff640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83b006aed0 0x7f83b006d390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83d01587f0 tx=0x7f83b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:43.804+0000 7f83c6ffd640 1 -- 172.21.15.31:0/2767659274 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f83c0054150 con 0x7f83d00cf570 2024-01-19T04:57:43.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:57:43.856 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:57:44.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.020+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f83d0074900 con 0x7f83d00cf570 2024-01-19T04:57:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.675+0000 7f83c6ffd640 1 -- 172.21.15.31:0/2767659274 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f83c0057770 con 0x7f83d00cf570 2024-01-19T04:57:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83b006aed0 msgr2=0x7f83b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83b006aed0 0x7f83b006d390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83d01587f0 tx=0x7f83b8009040 comp rx=0 tx=0).stop 2024-01-19T04:57:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d00cf570 msgr2=0x7f83d014b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d00cf570 0x7f83d014b4b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83c0004640 tx=0x7f83c000c8e0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 shutdown_connections 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f83b006aed0 0x7f83b006d390 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d0161860 0x7f83d00b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83d015d3a0 0x7f83d00b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 --2- 172.21.15.31:0/2767659274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83d00cf570 0x7f83d014b4b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 >> 172.21.15.31:0/2767659274 conn(0x7f83d00d1100 msgr2=0x7f83d00af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 shutdown_connections 2024-01-19T04:57:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.678+0000 7f83d4dde640 1 -- 172.21.15.31:0/2767659274 wait complete. 2024-01-19T04:57:44.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-19T04:57:44.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.803+0000 7faa1761d640 1 Processor -- start 2024-01-19T04:57:44.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.803+0000 7faa1761d640 1 -- start start 2024-01-19T04:57:44.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa1761d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa1015b1b0 0x7faa100cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:44.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa1761d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7faa1015f6b0 0x7faa100ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:44.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa1761d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa100cfb80 con 0x7faa1015b590 2024-01-19T04:57:44.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa1761d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa100cec90 con 0x7faa1015b1b0 2024-01-19T04:57:44.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa1761d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa100cef70 con 0x7faa1015f6b0 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa153b9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa1015b1b0 0x7faa100cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa153b9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa1015b1b0 0x7faa100cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54674/0 (socket says 172.21.15.31:54674) 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa153b9640 1 -- 172.21.15.31:0/517166743 learned_addr learned my addr 172.21.15.31:0/517166743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa15bba640 1 --2- 172.21.15.31:0/517166743 >> v2:172.21.15.137:3300/0 conn(0x7faa1015f6b0 0x7faa100ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa153b9640 1 -- 172.21.15.31:0/517166743 >> v2:172.21.15.137:3300/0 conn(0x7faa1015f6b0 msgr2=0x7faa100ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa153b9640 1 --2- 172.21.15.31:0/517166743 >> v2:172.21.15.137:3300/0 conn(0x7faa1015f6b0 0x7faa100ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa153b9640 1 -- 172.21.15.31:0/517166743 >> v1:172.21.15.137:6789/0 conn(0x7faa1015b590 legacy=0x7faa1015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa153b9640 1 -- 172.21.15.31:0/517166743 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100cf250 con 0x7faa1015b1b0 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.804+0000 7faa15bba640 1 --2- 172.21.15.31:0/517166743 >> v2:172.21.15.137:3300/0 conn(0x7faa1015f6b0 0x7faa100ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:44.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.805+0000 7faa153b9640 1 --2- 172.21.15.31:0/517166743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b1b0 0x7faa100cc800 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7faa100cfaf0 tx=0x7faa0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9703359bcb5eb98a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.805+0000 7faa0ffff640 1 -- 172.21.15.31:0/517166743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08031870 con 0x7faa1015b1b0 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.805+0000 7faa0ffff640 1 -- 172.21.15.31:0/517166743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa080319d0 con 0x7faa1015b1b0 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.805+0000 7faa0ffff640 1 -- 172.21.15.31:0/517166743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08031d60 con 0x7faa1015b1b0 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 -- 172.21.15.31:0/517166743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b1b0 msgr2=0x7faa100cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 --2- 172.21.15.31:0/517166743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b1b0 0x7faa100cc800 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7faa100cfaf0 tx=0x7faa0802b910 comp rx=0 tx=0).stop 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 -- 172.21.15.31:0/517166743 shutdown_connections 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 --2- 172.21.15.31:0/517166743 >> v2:172.21.15.137:3300/0 conn(0x7faa1015f6b0 0x7faa100ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 --2- 172.21.15.31:0/517166743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b1b0 0x7faa100cc800 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 -- 172.21.15.31:0/517166743 >> 172.21.15.31:0/517166743 conn(0x7faa100b2bb0 msgr2=0x7faa100b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 -- 172.21.15.31:0/517166743 shutdown_connections 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.806+0000 7faa1761d640 1 -- 172.21.15.31:0/517166743 wait complete. 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.807+0000 7faa1761d640 1 Processor -- start 2024-01-19T04:57:44.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.807+0000 7faa1761d640 1 -- start start 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa1761d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa1015b1b0 0x7faa10151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa1761d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b590 0x7faa10152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa1761d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa1015f6b0 0x7faa100b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa1761d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100b4970 con 0x7faa1015b590 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa1761d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa10156850 con 0x7faa1015f6b0 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa1761d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa10155f20 con 0x7faa1015b1b0 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b590 0x7faa10152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b590 0x7faa10152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54678/0 (socket says 172.21.15.31:54678) 2024-01-19T04:57:44.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 -- 172.21.15.31:0/3591930652 learned_addr learned my addr 172.21.15.31:0/3591930652 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa153b9640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa1015b1b0 0x7faa10151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa15bba640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa1015f6b0 0x7faa100b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 -- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa1015b1b0 msgr2=0x7faa10151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa1015b1b0 0x7faa10151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 -- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa1015f6b0 msgr2=0x7faa100b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa1015f6b0 0x7faa100b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.808+0000 7faa14bb8640 1 -- 172.21.15.31:0/3591930652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa080094d0 con 0x7faa1015b590 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.809+0000 7faa14bb8640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b590 0x7faa10152ef0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c430 tx=0x7faa0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.809+0000 7faa0dffb640 1 -- 172.21.15.31:0/3591930652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04011790 con 0x7faa1015b590 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.809+0000 7faa0dffb640 1 -- 172.21.15.31:0/3591930652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa040118f0 con 0x7faa1015b590 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.809+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa10156200 con 0x7faa1015b590 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.809+0000 7faa0dffb640 1 -- 172.21.15.31:0/3591930652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04011ce0 con 0x7faa1015b590 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.809+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa100b44e0 con 0x7faa1015b590 2024-01-19T04:57:44.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.810+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa100cfff0 con 0x7faa1015b590 2024-01-19T04:57:44.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.810+0000 7faa0dffb640 1 -- 172.21.15.31:0/3591930652 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7faa0401c460 con 0x7faa1015b590 2024-01-19T04:57:44.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.811+0000 7faa0dffb640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9fc06aed0 0x7fa9fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:44.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.811+0000 7faa0dffb640 1 -- 172.21.15.31:0/3591930652 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faa04089670 con 0x7faa1015b590 2024-01-19T04:57:44.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.811+0000 7faa153b9640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9fc06aed0 0x7fa9fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:44.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.812+0000 7faa153b9640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9fc06aed0 0x7fa9fc06d390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa08036000 tx=0x7faa0803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.815+0000 7faa0dffb640 1 -- 172.21.15.31:0/3591930652 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7faa100cfff0 con 0x7faa1015b590 2024-01-19T04:57:44.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:44.994+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faa100cfff0 con 0x7faa1015b590 2024-01-19T04:57:45.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.676+0000 7faa0dffb640 1 -- 172.21.15.31:0/3591930652 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7faa100cfff0 con 0x7faa1015b590 2024-01-19T04:57:45.676 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-01-19T04:57:45.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9fc06aed0 msgr2=0x7fa9fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:45.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9fc06aed0 0x7fa9fc06d390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa08036000 tx=0x7faa0803c040 comp rx=0 tx=0).stop 2024-01-19T04:57:45.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b590 msgr2=0x7faa10152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:45.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b590 0x7faa10152ef0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c430 tx=0x7faa0400c8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 shutdown_connections 2024-01-19T04:57:45.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa9fc06aed0 0x7fa9fc06d390 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa1015f6b0 0x7faa100b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1015b590 0x7faa10152ef0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 --2- 172.21.15.31:0/3591930652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faa1015b1b0 0x7faa10151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 >> 172.21.15.31:0/3591930652 conn(0x7faa100b2bb0 msgr2=0x7faa100b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:45.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 shutdown_connections 2024-01-19T04:57:45.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.680+0000 7faa1761d640 1 -- 172.21.15.31:0/3591930652 wait complete. 2024-01-19T04:57:45.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-19T04:57:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.806+0000 7f2f21384640 1 Processor -- start 2024-01-19T04:57:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.806+0000 7f2f21384640 1 -- start start 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f21384640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f1c06c560 0x7f2f1c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f21384640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c06bb40 con 0x7f2f1c06c940 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f21384640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c0d00a0 con 0x7f2f1c0cfb40 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f21384640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c0d0380 con 0x7f2f1c06c560 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f1affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f1c06c560 0x7f2f1c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f1a7fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2f1c06c940 0x7f2f1c0cf430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39564/0 (socket says 172.21.15.31:39564) 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f1a7fc640 1 -- 172.21.15.31:0/2840311195 learned_addr learned my addr 172.21.15.31:0/2840311195 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f1affd640 1 -- 172.21.15.31:0/2840311195 >> v1:172.21.15.137:6789/0 conn(0x7f2f1c0cfb40 legacy=0x7f2f1c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f1affd640 1 -- 172.21.15.31:0/2840311195 >> v1:172.21.15.31:6789/0 conn(0x7f2f1c06c940 legacy=0x7f2f1c0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f19ffb640 1 -- 172.21.15.31:0/2840311195 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76412282 0 0) 0x7f2f1c06bb40 con 0x7f2f1c06c940 2024-01-19T04:57:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.807+0000 7f2f1affd640 1 -- 172.21.15.31:0/2840311195 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0d0660 con 0x7f2f1c06c560 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.808+0000 7f2f1affd640 1 --2- 172.21.15.31:0/2840311195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c560 0x7f2f1c099a50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f140097a0 tx=0x7f2f1402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8eeff6461d943c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.808+0000 7f2f19ffb640 1 -- 172.21.15.31:0/2840311195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14030610 con 0x7f2f1c06c560 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.808+0000 7f2f19ffb640 1 -- 172.21.15.31:0/2840311195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14030770 con 0x7f2f1c06c560 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.808+0000 7f2f19ffb640 1 -- 172.21.15.31:0/2840311195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14030b00 con 0x7f2f1c06c560 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.809+0000 7f2f21384640 1 -- 172.21.15.31:0/2840311195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c560 msgr2=0x7f2f1c099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.809+0000 7f2f21384640 1 --2- 172.21.15.31:0/2840311195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c560 0x7f2f1c099a50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f140097a0 tx=0x7f2f1402b960 comp rx=0 tx=0).stop 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.809+0000 7f2f21384640 1 -- 172.21.15.31:0/2840311195 shutdown_connections 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.809+0000 7f2f21384640 1 --2- 172.21.15.31:0/2840311195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c560 0x7f2f1c099a50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.809+0000 7f2f21384640 1 -- 172.21.15.31:0/2840311195 >> 172.21.15.31:0/2840311195 conn(0x7f2f1c0b2bb0 msgr2=0x7f2f1c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.809+0000 7f2f21384640 1 -- 172.21.15.31:0/2840311195 shutdown_connections 2024-01-19T04:57:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.809+0000 7f2f21384640 1 -- 172.21.15.31:0/2840311195 wait complete. 2024-01-19T04:57:45.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.810+0000 7f2f21384640 1 Processor -- start 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.810+0000 7f2f21384640 1 -- start start 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f21384640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2f1c06c560 0x7f2f1c1e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f21384640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c940 0x7f2f1c1e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f21384640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f1c0cfb40 0x7f2f1c1e6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f21384640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c1e8130 con 0x7f2f1c06c940 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f21384640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c09ae20 con 0x7f2f1c0cfb40 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f21384640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c09b100 con 0x7f2f1c06c560 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c940 0x7f2f1c1e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c940 0x7f2f1c1e61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54690/0 (socket says 172.21.15.31:54690) 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 -- 172.21.15.31:0/2310867711 learned_addr learned my addr 172.21.15.31:0/2310867711 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:45.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 -- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2f1c06c560 msgr2=0x7f2f1c1e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2f1c06c560 0x7f2f1c1e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 -- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f1c0cfb40 msgr2=0x7f2f1c1e6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f1c0cfb40 0x7f2f1c1e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 -- 172.21.15.31:0/2310867711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14009400 con 0x7f2f1c06c940 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.811+0000 7f2f1a7fc640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c940 0x7f2f1c1e61d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400c3c0 tx=0x7f2f0400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.812+0000 7f2f137fe640 1 -- 172.21.15.31:0/2310867711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040074e0 con 0x7f2f1c06c940 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.812+0000 7f2f137fe640 1 -- 172.21.15.31:0/2310867711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f04007640 con 0x7f2f1c06c940 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.812+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c09b3e0 con 0x7f2f1c06c940 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.812+0000 7f2f137fe640 1 -- 172.21.15.31:0/2310867711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04004a20 con 0x7f2f1c06c940 2024-01-19T04:57:45.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.812+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c09b9a0 con 0x7f2f1c06c940 2024-01-19T04:57:45.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.813+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c0696e0 con 0x7f2f1c06c940 2024-01-19T04:57:45.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.813+0000 7f2f137fe640 1 -- 172.21.15.31:0/2310867711 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f2f04004b80 con 0x7f2f1c06c940 2024-01-19T04:57:45.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.813+0000 7f2f137fe640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2efc06aed0 0x7f2efc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:45.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.814+0000 7f2f137fe640 1 -- 172.21.15.31:0/2310867711 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f2f04089280 con 0x7f2f1c06c940 2024-01-19T04:57:45.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.814+0000 7f2f1affd640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2efc06aed0 0x7f2efc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:45.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.814+0000 7f2f1affd640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2efc06aed0 0x7f2efc06d390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2f14036000 tx=0x7f2f14030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:45.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:45.818+0000 7f2f137fe640 1 -- 172.21.15.31:0/2310867711 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f2f040542b0 con 0x7f2f1c06c940 2024-01-19T04:57:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.000+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2f1c0b3a70 con 0x7f2f1c06c940 2024-01-19T04:57:46.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.681+0000 7f2f137fe640 1 -- 172.21.15.31:0/2310867711 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2f040578d0 con 0x7f2f1c06c940 2024-01-19T04:57:46.681 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-01-19T04:57:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2efc06aed0 msgr2=0x7f2efc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2efc06aed0 0x7f2efc06d390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2f14036000 tx=0x7f2f14030000 comp rx=0 tx=0).stop 2024-01-19T04:57:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c940 msgr2=0x7f2f1c1e61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c940 0x7f2f1c1e61d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400c3c0 tx=0x7f2f0400c880 comp rx=0 tx=0).stop 2024-01-19T04:57:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 shutdown_connections 2024-01-19T04:57:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f2efc06aed0 0x7f2efc06d390 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f1c0cfb40 0x7f2f1c1e6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f1c06c940 0x7f2f1c1e61d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 --2- 172.21.15.31:0/2310867711 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2f1c06c560 0x7f2f1c1e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.684+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 >> 172.21.15.31:0/2310867711 conn(0x7f2f1c0b2bb0 msgr2=0x7f2f1c0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.685+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 shutdown_connections 2024-01-19T04:57:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.685+0000 7f2f21384640 1 -- 172.21.15.31:0/2310867711 wait complete. 2024-01-19T04:57:46.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-19T04:57:46.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.805+0000 7f74849fc640 1 Processor -- start 2024-01-19T04:57:46.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.805+0000 7f74849fc640 1 -- start start 2024-01-19T04:57:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.805+0000 7f74849fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74800d18e0 0x7f748015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.805+0000 7f74849fc640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f74800d3e50 0x7f74800d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f74849fc640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800d2200 con 0x7f74800d0ef0 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f74849fc640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748015dee0 con 0x7f74800d18e0 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f74849fc640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748015e1c0 con 0x7f74800d3e50 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f747e59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74800d18e0 0x7f748015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f747e59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74800d18e0 0x7f748015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54696/0 (socket says 172.21.15.31:54696) 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f747e59c640 1 -- 172.21.15.31:0/4262936407 learned_addr learned my addr 172.21.15.31:0/4262936407 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f747e59c640 1 -- 172.21.15.31:0/4262936407 >> v2:172.21.15.137:3300/0 conn(0x7f74800d3e50 msgr2=0x7f74800d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f747e59c640 1 --2- 172.21.15.31:0/4262936407 >> v2:172.21.15.137:3300/0 conn(0x7f74800d3e50 0x7f74800d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f747e59c640 1 -- 172.21.15.31:0/4262936407 >> v1:172.21.15.137:6789/0 conn(0x7f74800d0ef0 legacy=0x7f7480161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:46.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.806+0000 7f747e59c640 1 -- 172.21.15.31:0/4262936407 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748015e4a0 con 0x7f74800d18e0 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f747e59c640 1 --2- 172.21.15.31:0/4262936407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f748015d860 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f74800b5f10 tx=0x7f747002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab69d093b2548227 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f747dd9b640 1 -- 172.21.15.31:0/4262936407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470031870 con 0x7f74800d18e0 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f747dd9b640 1 -- 172.21.15.31:0/4262936407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74700319d0 con 0x7f74800d18e0 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f747dd9b640 1 -- 172.21.15.31:0/4262936407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470031d60 con 0x7f74800d18e0 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 -- 172.21.15.31:0/4262936407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 msgr2=0x7f748015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 --2- 172.21.15.31:0/4262936407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f748015d860 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f74800b5f10 tx=0x7f747002b910 comp rx=0 tx=0).stop 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 -- 172.21.15.31:0/4262936407 shutdown_connections 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 --2- 172.21.15.31:0/4262936407 >> v2:172.21.15.137:3300/0 conn(0x7f74800d3e50 0x7f74800d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 --2- 172.21.15.31:0/4262936407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f748015d860 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 -- 172.21.15.31:0/4262936407 >> 172.21.15.31:0/4262936407 conn(0x7f748009c600 msgr2=0x7f74800d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 -- 172.21.15.31:0/4262936407 shutdown_connections 2024-01-19T04:57:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.807+0000 7f74849fc640 1 -- 172.21.15.31:0/4262936407 wait complete. 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 Processor -- start 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 -- start start 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74800d0ef0 0x7f74800ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f74800c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74800d3e50 0x7f7480159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801f3450 con 0x7f74800d18e0 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800caa10 con 0x7f74800d0ef0 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.809+0000 7f74849fc640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c83e0 con 0x7f74800d3e50 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f74800c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f74800c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54704/0 (socket says 172.21.15.31:54704) 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 -- 172.21.15.31:0/2091847656 learned_addr learned my addr 172.21.15.31:0/2091847656 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f747e59c640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74800d0ef0 0x7f74800ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f747ed9d640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74800d3e50 0x7f7480159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 -- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74800d3e50 msgr2=0x7f7480159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74800d3e50 0x7f7480159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 -- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74800d0ef0 msgr2=0x7f74800ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74800d0ef0 0x7f74800ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 -- 172.21.15.31:0/2091847656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74700094d0 con 0x7f74800d18e0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f7475d9b640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f74800c7ea0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f746800b4a0 tx=0x7f746800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f74777fe640 1 -- 172.21.15.31:0/2091847656 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468010610 con 0x7f74800d18e0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f74777fe640 1 -- 172.21.15.31:0/2091847656 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7468010770 con 0x7f74800d18e0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74800c8610 con 0x7f74800d18e0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f74777fe640 1 -- 172.21.15.31:0/2091847656 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468010b20 con 0x7f74800d18e0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.810+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74800c89a0 con 0x7f74800d18e0 2024-01-19T04:57:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.811+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7480074900 con 0x7f74800d18e0 2024-01-19T04:57:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.812+0000 7f74777fe640 1 -- 172.21.15.31:0/2091847656 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7468010c80 con 0x7f74800d18e0 2024-01-19T04:57:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.812+0000 7f74777fe640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f746006aed0 0x7f746006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.812+0000 7f74777fe640 1 -- 172.21.15.31:0/2091847656 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7468088f00 con 0x7f74800d18e0 2024-01-19T04:57:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.813+0000 7f747e59c640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f746006aed0 0x7f746006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.813+0000 7f747e59c640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f746006aed0 0x7f746006d390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7470036000 tx=0x7f747003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.817+0000 7f74777fe640 1 -- 172.21.15.31:0/2091847656 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f74680540f0 con 0x7f74800d18e0 2024-01-19T04:57:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:46.998+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f74800d09a0 con 0x7f74800d18e0 2024-01-19T04:57:47.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.001+0000 7f74777fe640 1 -- 172.21.15.31:0/2091847656 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f746801e0a0 con 0x7f74800d18e0 2024-01-19T04:57:47.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f746006aed0 msgr2=0x7f746006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f746006aed0 0x7f746006d390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7470036000 tx=0x7f747003c040 comp rx=0 tx=0).stop 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 msgr2=0x7f74800c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f74800c7ea0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f746800b4a0 tx=0x7f746800b960 comp rx=0 tx=0).stop 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 shutdown_connections 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f746006aed0 0x7f746006d390 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f74800d3e50 0x7f7480159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74800d18e0 0x7f74800c7ea0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 --2- 172.21.15.31:0/2091847656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74800d0ef0 0x7f74800ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 >> 172.21.15.31:0/2091847656 conn(0x7f748009c600 msgr2=0x7f748006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 shutdown_connections 2024-01-19T04:57:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.004+0000 7f74849fc640 1 -- 172.21.15.31:0/2091847656 wait complete. 2024-01-19T04:57:47.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-01-19T04:57:47.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.115+0000 7fa32ba10640 1 Processor -- start 2024-01-19T04:57:47.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa32ba10640 1 -- start start 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa32ba10640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa324160a10 0x7fa3240b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa32ba10640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240b4d00 0x7fa3240b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa32ba10640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa324160df0 con 0x7fa3240d2e50 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa32ba10640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3240b7300 con 0x7fa324160a10 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa32ba10640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240b5660 con 0x7fa3240b4d00 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa3297ac640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa324160a10 0x7fa3240b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa3297ac640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa324160a10 0x7fa3240b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54706/0 (socket says 172.21.15.31:54706) 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.116+0000 7fa3297ac640 1 -- 172.21.15.31:0/4100082655 learned_addr learned my addr 172.21.15.31:0/4100082655 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.117+0000 7fa3297ac640 1 -- 172.21.15.31:0/4100082655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240b4d00 msgr2=0x7fa3240b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.117+0000 7fa329fad640 1 --2- 172.21.15.31:0/4100082655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240b4d00 0x7fa3240b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.117+0000 7fa3297ac640 1 --2- 172.21.15.31:0/4100082655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240b4d00 0x7fa3240b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.117+0000 7fa3297ac640 1 -- 172.21.15.31:0/4100082655 >> v1:172.21.15.137:6789/0 conn(0x7fa3240d2e50 legacy=0x7fa3240d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.117+0000 7fa3297ac640 1 -- 172.21.15.31:0/4100082655 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240d1db0 con 0x7fa324160a10 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.117+0000 7fa329fad640 1 --2- 172.21.15.31:0/4100082655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240b4d00 0x7fa3240b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa3297ac640 1 --2- 172.21.15.31:0/4100082655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324160a10 0x7fa3240b8ea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa320009870 tx=0x7fa32002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df49adbf3f5d5a server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa31bfff640 1 -- 172.21.15.31:0/4100082655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320030840 con 0x7fa324160a10 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa31bfff640 1 -- 172.21.15.31:0/4100082655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200309a0 con 0x7fa324160a10 2024-01-19T04:57:47.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa31bfff640 1 -- 172.21.15.31:0/4100082655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320030d30 con 0x7fa324160a10 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 -- 172.21.15.31:0/4100082655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324160a10 msgr2=0x7fa3240b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 --2- 172.21.15.31:0/4100082655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324160a10 0x7fa3240b8ea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa320009870 tx=0x7fa32002b980 comp rx=0 tx=0).stop 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 -- 172.21.15.31:0/4100082655 shutdown_connections 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 --2- 172.21.15.31:0/4100082655 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240b4d00 0x7fa3240b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 --2- 172.21.15.31:0/4100082655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324160a10 0x7fa3240b8ea0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 -- 172.21.15.31:0/4100082655 >> 172.21.15.31:0/4100082655 conn(0x7fa324072310 msgr2=0x7fa3240cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 -- 172.21.15.31:0/4100082655 shutdown_connections 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.118+0000 7fa32ba10640 1 -- 172.21.15.31:0/4100082655 wait complete. 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.119+0000 7fa32ba10640 1 Processor -- start 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa32ba10640 1 -- start start 2024-01-19T04:57:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa32ba10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240b4d00 0x7fa3240c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa32ba10640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240d2e50 0x7fa3241f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa32ba10640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa324160a10 0x7fa324154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa32ba10640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241f7710 con 0x7fa3240b4d00 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa32ba10640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240cbb80 con 0x7fa324160a10 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa32ba10640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240c96e0 con 0x7fa3240d2e50 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa3297ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240b4d00 0x7fa3240c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa3297ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240b4d00 0x7fa3240c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54710/0 (socket says 172.21.15.31:54710) 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa3297ac640 1 -- 172.21.15.31:0/2627365565 learned_addr learned my addr 172.21.15.31:0/2627365565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.120+0000 7fa328fab640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240d2e50 0x7fa3241f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa329fad640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa324160a10 0x7fa324154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa3297ac640 1 -- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240d2e50 msgr2=0x7fa3241f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa3297ac640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240d2e50 0x7fa3241f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa3297ac640 1 -- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa324160a10 msgr2=0x7fa324154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa3297ac640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa324160a10 0x7fa324154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa3297ac640 1 -- 172.21.15.31:0/2627365565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3200094d0 con 0x7fa3240b4d00 2024-01-19T04:57:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa3297ac640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240b4d00 0x7fa3240c6db0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa320009870 tx=0x7fa320004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa319ffb640 1 -- 172.21.15.31:0/2627365565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200307f0 con 0x7fa3240b4d00 2024-01-19T04:57:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa319ffb640 1 -- 172.21.15.31:0/2627365565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320039d90 con 0x7fa3240b4d00 2024-01-19T04:57:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa319ffb640 1 -- 172.21.15.31:0/2627365565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320043630 con 0x7fa3240b4d00 2024-01-19T04:57:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3240c99c0 con 0x7fa3240b4d00 2024-01-19T04:57:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.121+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3240cc100 con 0x7fa3240b4d00 2024-01-19T04:57:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.122+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3240b9270 con 0x7fa3240b4d00 2024-01-19T04:57:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.123+0000 7fa319ffb640 1 -- 172.21.15.31:0/2627365565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fa32004c3b0 con 0x7fa3240b4d00 2024-01-19T04:57:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.123+0000 7fa319ffb640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa30806aed0 0x7fa30806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.123+0000 7fa319ffb640 1 -- 172.21.15.31:0/2627365565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa3200ab860 con 0x7fa3240b4d00 2024-01-19T04:57:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.123+0000 7fa328fab640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa30806aed0 0x7fa30806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.124+0000 7fa328fab640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa30806aed0 0x7fa30806d390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa3240d0ae0 tx=0x7fa314009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:47.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.127+0000 7fa319ffb640 1 -- 172.21.15.31:0/2627365565 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa320076a30 con 0x7fa3240b4d00 2024-01-19T04:57:47.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.375+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fa3240ca670 con 0x7fa3240b4d00 2024-01-19T04:57:47.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.682+0000 7fa319ffb640 1 -- 172.21.15.31:0/2627365565 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v154) v1 ==== 134+0+0 (secure 0 0 0) 0x7fa3200480c0 con 0x7fa3240b4d00 2024-01-19T04:57:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa30806aed0 msgr2=0x7fa30806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa30806aed0 0x7fa30806d390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa3240d0ae0 tx=0x7fa314009040 comp rx=0 tx=0).stop 2024-01-19T04:57:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240b4d00 msgr2=0x7fa3240c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240b4d00 0x7fa3240c6db0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa320009870 tx=0x7fa320004730 comp rx=0 tx=0).stop 2024-01-19T04:57:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 shutdown_connections 2024-01-19T04:57:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fa30806aed0 0x7fa30806d390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa324160a10 0x7fa324154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa3240d2e50 0x7fa3241f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.684+0000 7fa32ba10640 1 --2- 172.21.15.31:0/2627365565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240b4d00 0x7fa3240c6db0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.685+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 >> 172.21.15.31:0/2627365565 conn(0x7fa324072310 msgr2=0x7fa3240ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.685+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 shutdown_connections 2024-01-19T04:57:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:47.685+0000 7fa32ba10640 1 -- 172.21.15.31:0/2627365565 wait complete. 2024-01-19T04:57:47.696 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa/tasks/cephfs/mount.py", line 775, in run_python 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: p.wait() 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-19T04:57:47.697 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner: 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:Ran 2 tests in 118.292s 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner: 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner: 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-19T04:57:47.698 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa/tasks/cephfs/mount.py", line 775, in run_python 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: p.wait() 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-01-19T04:57:47.699 INFO:tasks.cephfs_test_runner: 2024-01-19T04:57:47.699 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-19T04:57:47.928 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c00d74b8f14945ddbe0d34c44d48a4ab Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0edf41a622739843d6b978b179ff3227b476dd9d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-19T04:57:47.930 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-01-19T04:57:47.939 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-01-19T04:57:47.956 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-01-19T04:57:47.956 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:47.956 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:47.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057151,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20760 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249472k,nr_inodes=812368,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-19T04:57:47.984 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-19T04:57:47.984 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:48.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057151,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20760 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249472k,nr_inodes=812368,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498952k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-19T04:57:48.041 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:57:48.041 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:48.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:57:48.060 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-19T04:57:48.060 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-19T04:57:48.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2326 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-19T04:57:48.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.118 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-01-19T04:57:48.176 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-19T04:57:48.176 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-19T04:57:48.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.177 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-19T04:57:48.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.248 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-19T04:57:48.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.320 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link delete ceph-brx 2024-01-19T04:57:48.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.421 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-01-19T04:57:48.438 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-19T04:57:48.438 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-19T04:57:48.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-19T04:57:48.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-19T04:57:48.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-19T04:57:48.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-01-19T04:57:48.679 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-19T04:57:48.690 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-19T04:57:48.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:57:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.794+0000 7ff0a3e1d640 1 Processor -- start 2024-01-19T04:57:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.795+0000 7ff0a3e1d640 1 -- start start 2024-01-19T04:57:48.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.795+0000 7ff0a3e1d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff09c150a50 0x7ff09c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:48.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.795+0000 7ff0a3e1d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff09c0ce550 0x7ff09c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.795+0000 7ff0a3e1d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c070e50 con 0x7ff09c0b2f90 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.795+0000 7ff0a3e1d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0cba30 con 0x7ff09c0ce550 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.795+0000 7ff0a3e1d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0b33a0 con 0x7ff09c150a50 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.795+0000 7ff0a13b8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff09c0ce550 0x7ff09c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.796+0000 7ff0a23ba640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff09c0b2f90 0x7ff09c15d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39578/0 (socket says 172.21.15.31:39578) 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.796+0000 7ff0a23ba640 1 -- 172.21.15.31:0/2782399560 learned_addr learned my addr 172.21.15.31:0/2782399560 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.796+0000 7ff0a1bb9640 1 --2- 172.21.15.31:0/2782399560 >> v2:172.21.15.137:3300/0 conn(0x7ff09c150a50 0x7ff09c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:48.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.796+0000 7ff0a13b8640 1 -- 172.21.15.31:0/2782399560 >> v2:172.21.15.137:3300/0 conn(0x7ff09c150a50 msgr2=0x7ff09c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:48.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.796+0000 7ff0a13b8640 1 --2- 172.21.15.31:0/2782399560 >> v2:172.21.15.137:3300/0 conn(0x7ff09c150a50 0x7ff09c159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.796+0000 7ff0a13b8640 1 -- 172.21.15.31:0/2782399560 >> v1:172.21.15.31:6789/0 conn(0x7ff09c0b2f90 legacy=0x7ff09c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:48.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.796+0000 7ff0a13b8640 1 -- 172.21.15.31:0/2782399560 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c15a4f0 con 0x7ff09c0ce550 2024-01-19T04:57:48.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a13b8640 1 --2- 172.21.15.31:0/2782399560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0ce550 0x7ff09c0b2a50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff09c06e140 tx=0x7ff08c02c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb98052b84a0399 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:48.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a0bb7640 1 -- 172.21.15.31:0/2782399560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c030800 con 0x7ff09c0ce550 2024-01-19T04:57:48.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a0bb7640 1 -- 172.21.15.31:0/2782399560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c030960 con 0x7ff09c0ce550 2024-01-19T04:57:48.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a0bb7640 1 -- 172.21.15.31:0/2782399560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c030cf0 con 0x7ff09c0ce550 2024-01-19T04:57:48.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2782399560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0ce550 msgr2=0x7ff09c0b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:48.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2782399560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0ce550 0x7ff09c0b2a50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff09c06e140 tx=0x7ff08c02c9e0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2782399560 shutdown_connections 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2782399560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0ce550 0x7ff09c0b2a50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2782399560 >> v2:172.21.15.137:3300/0 conn(0x7ff09c150a50 0x7ff09c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2782399560 >> 172.21.15.31:0/2782399560 conn(0x7ff09c06a790 msgr2=0x7ff09c06aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2782399560 shutdown_connections 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.797+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2782399560 wait complete. 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.798+0000 7ff0a3e1d640 1 Processor -- start 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.798+0000 7ff0a3e1d640 1 -- start start 2024-01-19T04:57:48.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a3e1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0b2f90 0x7ff09c141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a3e1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09c0ce550 0x7ff09c06cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a3e1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff09c150a50 0x7ff09c06d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a3e1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0afdb0 con 0x7ff09c0b2f90 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a3e1d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0b0090 con 0x7ff09c0ce550 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a3e1d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1e3e60 con 0x7ff09c150a50 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a1bb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0b2f90 0x7ff09c141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a1bb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0b2f90 0x7ff09c141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54722/0 (socket says 172.21.15.31:54722) 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.799+0000 7ff0a1bb9640 1 -- 172.21.15.31:0/2822698076 learned_addr learned my addr 172.21.15.31:0/2822698076 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:48.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a23ba640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff09c150a50 0x7ff09c06d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a13b8640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09c0ce550 0x7ff09c06cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a1bb9640 1 -- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff09c150a50 msgr2=0x7ff09c06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a1bb9640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff09c150a50 0x7ff09c06d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a1bb9640 1 -- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09c0ce550 msgr2=0x7ff09c06cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a1bb9640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09c0ce550 0x7ff09c06cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a1bb9640 1 -- 172.21.15.31:0/2822698076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c009400 con 0x7ff09c0b2f90 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a23ba640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff09c150a50 0x7ff09c06d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a13b8640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09c0ce550 0x7ff09c06cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a1bb9640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0b2f90 0x7ff09c141d20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff09800e9b0 tx=0x7ff09800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0927fc640 1 -- 172.21.15.31:0/2822698076 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098009670 con 0x7ff09c0b2f90 2024-01-19T04:57:48.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0927fc640 1 -- 172.21.15.31:0/2822698076 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09800c9a0 con 0x7ff09c0b2f90 2024-01-19T04:57:48.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c1e4140 con 0x7ff09c0b2f90 2024-01-19T04:57:48.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0927fc640 1 -- 172.21.15.31:0/2822698076 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800cd50 con 0x7ff09c0b2f90 2024-01-19T04:57:48.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.800+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c14e0a0 con 0x7ff09c0b2f90 2024-01-19T04:57:48.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.801+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c0696e0 con 0x7ff09c0b2f90 2024-01-19T04:57:48.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.801+0000 7ff0927fc640 1 -- 172.21.15.31:0/2822698076 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff098013470 con 0x7ff09c0b2f90 2024-01-19T04:57:48.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.802+0000 7ff0927fc640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff08006aed0 0x7ff08006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:48.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.802+0000 7ff0927fc640 1 -- 172.21.15.31:0/2822698076 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff098089590 con 0x7ff09c0b2f90 2024-01-19T04:57:48.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.802+0000 7ff0a13b8640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff08006aed0 0x7ff08006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:48.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.802+0000 7ff0a13b8640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff08006aed0 0x7ff08006d390 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff08c037000 tx=0x7ff08c03a040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:48.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.806+0000 7ff0927fc640 1 -- 172.21.15.31:0/2822698076 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff098054760 con 0x7ff09c0b2f90 2024-01-19T04:57:48.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.979+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff09c15a780 con 0x7ff08006aed0 2024-01-19T04:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.983+0000 7ff0927fc640 1 -- 172.21.15.31:0/2822698076 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51235 (secure 0 0 0) 0x7ff08c030b50 con 0x7ff08006aed0 2024-01-19T04:57:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:48.983 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:57:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff08006aed0 msgr2=0x7ff08006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff08006aed0 0x7ff08006d390 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff08c037000 tx=0x7ff08c03a040 comp rx=0 tx=0).stop 2024-01-19T04:57:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0b2f90 msgr2=0x7ff09c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0b2f90 0x7ff09c141d20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff09800e9b0 tx=0x7ff09800ee70 comp rx=0 tx=0).stop 2024-01-19T04:57:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 shutdown_connections 2024-01-19T04:57:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff08006aed0 0x7ff08006d390 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff09c150a50 0x7ff09c06d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09c0ce550 0x7ff09c06cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 --2- 172.21.15.31:0/2822698076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff09c0b2f90 0x7ff09c141d20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 >> 172.21.15.31:0/2822698076 conn(0x7ff09c06a790 msgr2=0x7ff09c0cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 shutdown_connections 2024-01-19T04:57:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:48.984+0000 7ff0a3e1d640 1 -- 172.21.15.31:0/2822698076 wait complete. 2024-01-19T04:57:48.994 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-01-19T04:57:47.596341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222184,"kb_used_data":6016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752536,"statfs":{"total":773094113280,"available":772866596864,"internally_reserved":0,"allocated":6160384,"data_stored":3249833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001470"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-01-19T04:57:44.004901+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:57:44.004901+0000","last_peered":"2024-01-19T04:57:44.004901+0000","last_clean":"2024-01-19T04:57:44.004901+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:57:44.004901+0000","last_undegraded":"2024-01-19T04:57:44.004901+0000","last_fullsized":"2024-01-19T04:57:44.004901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":849674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73099999999999998,"5min":0.753,"15min":0.753},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68200000000000005,"5min":0.745,"15min":0.745},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68500000000000005,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":253955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.312,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.69999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.59899999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68799999999999994,"5min":0.876,"15min":0.876},"last":0.496}]},{"osd":3,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.69499999999999995,"5min":0.754,"15min":0.754},"last":0.435}]},{"osd":4,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.877,"15min":0.877},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.372}]},{"osd":7,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92800000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.504,"15min":0.505},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.496},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.624},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.51300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98299999999999998,"5min":1.103,"15min":1.103},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84799999999999998,"5min":1.2,"15min":1.2},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0049999999999999,"5min":1.321,"15min":1.321},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":1.046,"15min":1.046},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.48499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0309999999999999,"5min":1.258,"15min":1.258},"last":1.052}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.622}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.442},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.498}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.318,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53200000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.88,"15min":0.88},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.755,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80300000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71299999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81999999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67700000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.248},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":1.47,"15min":1.47},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":1.006,"15min":1.006},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90900000000000003,"5min":1.355,"15min":1.355},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82999999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81899999999999995,"5min":0.995,"15min":0.995},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56299999999999994,"5min":1.397,"15min":1.397},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.23799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":1.423,"15min":1.423},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66700000000000004,"5min":1.446,"15min":1.446},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.432},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":1.619,"15min":1.619},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":1.585,"15min":1.585},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":1.333,"15min":1.333},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83899999999999997,"5min":1.367,"15min":1.367},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.624},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":849674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.36899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46400000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38200000000000001}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.42299999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81200000000000006,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.044,"15min":1.044},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:57:48.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:57:49.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.097+0000 7f6bb5f25640 1 Processor -- start 2024-01-19T04:57:49.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.097+0000 7f6bb5f25640 1 -- start start 2024-01-19T04:57:49.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6bb5f25640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb00b3450 0x7f6bb0143930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6bb5f25640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6bb0142320 0x7f6bb0143e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6bb5f25640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb0142890 con 0x7f6bb00cc230 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6bb5f25640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb0142b70 con 0x7f6bb0142320 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6bb5f25640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0142e50 con 0x7f6bb00b3450 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6bb0142320 0x7f6bb0143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baf7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6bb00cc230 0x7f6bb00b2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39582/0 (socket says 172.21.15.31:39582) 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baf7fe640 1 -- 172.21.15.31:0/3489952801 learned_addr learned my addr 172.21.15.31:0/3489952801 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baeffd640 1 --2- 172.21.15.31:0/3489952801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb00b3450 0x7f6bb0143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baffff640 1 -- 172.21.15.31:0/3489952801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb00b3450 msgr2=0x7f6bb0143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baffff640 1 --2- 172.21.15.31:0/3489952801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb00b3450 0x7f6bb0143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baffff640 1 -- 172.21.15.31:0/3489952801 >> v1:172.21.15.31:6789/0 conn(0x7f6bb00cc230 legacy=0x7f6bb00b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6baffff640 1 -- 172.21.15.31:0/3489952801 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0143130 con 0x7f6bb0142320 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.098+0000 7f6bae7fc640 1 -- 172.21.15.31:0/3489952801 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986046491 0 0) 0x7f6bb0142890 con 0x7f6bb00cc230 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.099+0000 7f6baffff640 1 --2- 172.21.15.31:0/3489952801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb0142320 0x7f6bb0143e70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009d90 tx=0x7f6ba802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3e2dd3fb6ee58 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.099+0000 7f6bae7fc640 1 -- 172.21.15.31:0/3489952801 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba802cc50 con 0x7f6bb0142320 2024-01-19T04:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.099+0000 7f6bae7fc640 1 -- 172.21.15.31:0/3489952801 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba802cdb0 con 0x7f6bb0142320 2024-01-19T04:57:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.099+0000 7f6bae7fc640 1 -- 172.21.15.31:0/3489952801 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8034660 con 0x7f6bb0142320 2024-01-19T04:57:49.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 -- 172.21.15.31:0/3489952801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb0142320 msgr2=0x7f6bb0143e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/3489952801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb0142320 0x7f6bb0143e70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009d90 tx=0x7f6ba802bea0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 -- 172.21.15.31:0/3489952801 shutdown_connections 2024-01-19T04:57:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/3489952801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb0142320 0x7f6bb0143e70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/3489952801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb00b3450 0x7f6bb0143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 -- 172.21.15.31:0/3489952801 >> 172.21.15.31:0/3489952801 conn(0x7f6bb006abe0 msgr2=0x7f6bb0066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 -- 172.21.15.31:0/3489952801 shutdown_connections 2024-01-19T04:57:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.100+0000 7f6bb5f25640 1 -- 172.21.15.31:0/3489952801 wait complete. 2024-01-19T04:57:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.101+0000 7f6bb5f25640 1 Processor -- start 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.101+0000 7f6bb5f25640 1 -- start start 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6bb5f25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb00b3450 0x7f6bb01f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6bb5f25640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb00cc230 0x7f6bb015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6bb5f25640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb0142320 0x7f6bb0158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6bb5f25640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb015edc0 con 0x7f6bb00cc230 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6bb5f25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb015d0c0 con 0x7f6bb00b3450 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6bb5f25640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb014cf90 con 0x7f6bb0142320 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb00cc230 0x7f6bb015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb00cc230 0x7f6bb015e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54734/0 (socket says 172.21.15.31:54734) 2024-01-19T04:57:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 -- 172.21.15.31:0/4057305820 learned_addr learned my addr 172.21.15.31:0/4057305820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baffff640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb0142320 0x7f6bb0158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baf7fe640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb00b3450 0x7f6bb01f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 -- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb0142320 msgr2=0x7f6bb0158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb0142320 0x7f6bb0158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 -- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb00b3450 msgr2=0x7f6bb01f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb00b3450 0x7f6bb01f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baeffd640 1 -- 172.21.15.31:0/4057305820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8009a70 con 0x7f6bb00cc230 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baffff640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb0142320 0x7f6bb0158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.102+0000 7f6baf7fe640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb00b3450 0x7f6bb01f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.103+0000 7f6baeffd640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb00cc230 0x7f6bb015e490 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400d800 tx=0x7f6ba400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.103+0000 7f6b97fff640 1 -- 172.21.15.31:0/4057305820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400b6c0 con 0x7f6bb00cc230 2024-01-19T04:57:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.103+0000 7f6b97fff640 1 -- 172.21.15.31:0/4057305820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba400b820 con 0x7f6bb00cc230 2024-01-19T04:57:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.103+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb014c660 con 0x7f6bb00cc230 2024-01-19T04:57:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.103+0000 7f6b97fff640 1 -- 172.21.15.31:0/4057305820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400bbb0 con 0x7f6bb00cc230 2024-01-19T04:57:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.103+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb014c9f0 con 0x7f6bb00cc230 2024-01-19T04:57:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.104+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb014c7f0 con 0x7f6bb00cc230 2024-01-19T04:57:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.104+0000 7f6b97fff640 1 -- 172.21.15.31:0/4057305820 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6ba400bd10 con 0x7f6bb00cc230 2024-01-19T04:57:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.104+0000 7f6b97fff640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b9c06aed0 0x7f6b9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.105+0000 7f6b97fff640 1 -- 172.21.15.31:0/4057305820 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6ba4089840 con 0x7f6bb00cc230 2024-01-19T04:57:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.105+0000 7f6baf7fe640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b9c06aed0 0x7f6b9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.105+0000 7f6baf7fe640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b9c06aed0 0x7f6b9c06d390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0000f60 tx=0x7f6ba0007c10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.109+0000 7f6b97fff640 1 -- 172.21.15.31:0/4057305820 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6ba4054aa0 con 0x7f6bb00cc230 2024-01-19T04:57:49.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.286+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bb015f000 con 0x7f6b9c06aed0 2024-01-19T04:57:49.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.289+0000 7f6b97fff640 1 -- 172.21.15.31:0/4057305820 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51235 (secure 0 0 0) 0x7f6bb015f000 con 0x7f6b9c06aed0 2024-01-19T04:57:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:49.289 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:57:49.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b9c06aed0 msgr2=0x7f6b9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b9c06aed0 0x7f6b9c06d390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0000f60 tx=0x7f6ba0007c10 comp rx=0 tx=0).stop 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb00cc230 msgr2=0x7f6bb015e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb00cc230 0x7f6bb015e490 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400d800 tx=0x7f6ba400dcc0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 shutdown_connections 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6b9c06aed0 0x7f6b9c06d390 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6bb0142320 0x7f6bb0158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bb00cc230 0x7f6bb015e490 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 --2- 172.21.15.31:0/4057305820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb00b3450 0x7f6bb01f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 >> 172.21.15.31:0/4057305820 conn(0x7f6bb006abe0 msgr2=0x7f6bb00b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:49.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 shutdown_connections 2024-01-19T04:57:49.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.291+0000 7f6bb5f25640 1 -- 172.21.15.31:0/4057305820 wait complete. 2024-01-19T04:57:49.303 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-01-19T04:57:47.596341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222184,"kb_used_data":6016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752536,"statfs":{"total":773094113280,"available":772866596864,"internally_reserved":0,"allocated":6160384,"data_stored":3249833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001470"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-01-19T04:57:44.004901+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:57:44.004901+0000","last_peered":"2024-01-19T04:57:44.004901+0000","last_clean":"2024-01-19T04:57:44.004901+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:57:44.004901+0000","last_undegraded":"2024-01-19T04:57:44.004901+0000","last_fullsized":"2024-01-19T04:57:44.004901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":849674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73099999999999998,"5min":0.753,"15min":0.753},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68200000000000005,"5min":0.745,"15min":0.745},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68500000000000005,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":253955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.312,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.69999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.59899999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68799999999999994,"5min":0.876,"15min":0.876},"last":0.496}]},{"osd":3,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.69499999999999995,"5min":0.754,"15min":0.754},"last":0.435}]},{"osd":4,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.877,"15min":0.877},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.372}]},{"osd":7,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92800000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.504,"15min":0.505},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.496},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.624},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.51300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98299999999999998,"5min":1.103,"15min":1.103},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84799999999999998,"5min":1.2,"15min":1.2},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0049999999999999,"5min":1.321,"15min":1.321},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":1.046,"15min":1.046},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.48499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0309999999999999,"5min":1.258,"15min":1.258},"last":1.052}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.622}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.442},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.498}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.318,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53200000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.88,"15min":0.88},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.755,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80300000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71299999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81999999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67700000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.248},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":1.47,"15min":1.47},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":1.006,"15min":1.006},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90900000000000003,"5min":1.355,"15min":1.355},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82999999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81899999999999995,"5min":0.995,"15min":0.995},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56299999999999994,"5min":1.397,"15min":1.397},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.23799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":1.423,"15min":1.423},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66700000000000004,"5min":1.446,"15min":1.446},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.432},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":1.619,"15min":1.619},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":1.585,"15min":1.585},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":1.333,"15min":1.333},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83899999999999997,"5min":1.367,"15min":1.367},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.624},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":849674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.36899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46400000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38200000000000001}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.42299999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81200000000000006,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.044,"15min":1.044},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:57:49.305 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-19T04:57:49.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:57:49.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.406+0000 7f4a3b448640 1 Processor -- start 2024-01-19T04:57:49.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.406+0000 7f4a3b448640 1 -- start start 2024-01-19T04:57:49.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.407+0000 7f4a3b448640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a34160bb0 0x7f4a340b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.407+0000 7f4a3b448640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4a340cfd10 0x7f4a340d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.407+0000 7f4a3b448640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a340d06b0 con 0x7f4a340d0ef0 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.407+0000 7f4a3b448640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a340b66b0 con 0x7f4a340cfd10 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.407+0000 7f4a3b448640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340d1b90 con 0x7f4a34160bb0 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a391e4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a34160bb0 0x7f4a340b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a389e3640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4a340cfd10 0x7f4a340d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a391e4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a34160bb0 0x7f4a340b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:44744/0 (socket says 172.21.15.31:44744) 2024-01-19T04:57:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a391e4640 1 -- 172.21.15.31:0/1284677662 learned_addr learned my addr 172.21.15.31:0/1284677662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a389e3640 1 -- 172.21.15.31:0/1284677662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a34160bb0 msgr2=0x7f4a340b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a389e3640 1 --2- 172.21.15.31:0/1284677662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a34160bb0 0x7f4a340b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a389e3640 1 -- 172.21.15.31:0/1284677662 >> v1:172.21.15.137:6789/0 conn(0x7f4a340d0ef0 legacy=0x7f4a340d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a389e3640 1 -- 172.21.15.31:0/1284677662 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340d1e70 con 0x7f4a340cfd10 2024-01-19T04:57:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.408+0000 7f4a33fff640 1 -- 172.21.15.31:0/1284677662 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395326665 0 0) 0x7f4a340d06b0 con 0x7f4a340d0ef0 2024-01-19T04:57:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.409+0000 7f4a389e3640 1 --2- 172.21.15.31:0/1284677662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a340cfd10 0x7f4a340d0170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a280097a0 tx=0x7f4a2802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ab476adf1025d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.409+0000 7f4a33fff640 1 -- 172.21.15.31:0/1284677662 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28031830 con 0x7f4a340cfd10 2024-01-19T04:57:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.409+0000 7f4a33fff640 1 -- 172.21.15.31:0/1284677662 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28031990 con 0x7f4a340cfd10 2024-01-19T04:57:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.409+0000 7f4a33fff640 1 -- 172.21.15.31:0/1284677662 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28031d20 con 0x7f4a340cfd10 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 -- 172.21.15.31:0/1284677662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a340cfd10 msgr2=0x7f4a340d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1284677662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a340cfd10 0x7f4a340d0170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a280097a0 tx=0x7f4a2802b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 -- 172.21.15.31:0/1284677662 shutdown_connections 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1284677662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a340cfd10 0x7f4a340d0170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1284677662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a34160bb0 0x7f4a340b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 -- 172.21.15.31:0/1284677662 >> 172.21.15.31:0/1284677662 conn(0x7f4a3409c600 msgr2=0x7f4a340d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 -- 172.21.15.31:0/1284677662 shutdown_connections 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.410+0000 7f4a3b448640 1 -- 172.21.15.31:0/1284677662 wait complete. 2024-01-19T04:57:49.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.411+0000 7f4a3b448640 1 Processor -- start 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.411+0000 7f4a3b448640 1 -- start start 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a3b448640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a340cfd10 0x7f4a341f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a3b448640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a340d0ef0 0x7f4a340c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a3b448640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a34160bb0 0x7f4a340c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a3b448640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a341604b0 con 0x7f4a340d0ef0 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a3b448640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340c72d0 con 0x7f4a34160bb0 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a3b448640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340cd3c0 con 0x7f4a340cfd10 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a340d0ef0 0x7f4a340c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a340d0ef0 0x7f4a340c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54738/0 (socket says 172.21.15.31:54738) 2024-01-19T04:57:49.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 -- 172.21.15.31:0/1909977988 learned_addr learned my addr 172.21.15.31:0/1909977988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a391e4640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a340cfd10 0x7f4a341f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a399e5640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a34160bb0 0x7f4a340c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 -- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a340cfd10 msgr2=0x7f4a341f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a340cfd10 0x7f4a341f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 -- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a34160bb0 msgr2=0x7f4a340c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a34160bb0 0x7f4a340c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 -- 172.21.15.31:0/1909977988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28009400 con 0x7f4a340d0ef0 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a391e4640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a340cfd10 0x7f4a341f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a389e3640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a340d0ef0 0x7f4a340c94c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034040 tx=0x7f4a280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.412+0000 7f4a31ffb640 1 -- 172.21.15.31:0/1909977988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2803a8f0 con 0x7f4a340d0ef0 2024-01-19T04:57:49.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.413+0000 7f4a31ffb640 1 -- 172.21.15.31:0/1909977988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2803aa70 con 0x7f4a340d0ef0 2024-01-19T04:57:49.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.413+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3414fd80 con 0x7f4a340d0ef0 2024-01-19T04:57:49.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.413+0000 7f4a31ffb640 1 -- 172.21.15.31:0/1909977988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2803ae20 con 0x7f4a340d0ef0 2024-01-19T04:57:49.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.413+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a340c6ef0 con 0x7f4a340d0ef0 2024-01-19T04:57:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.414+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a340c7410 con 0x7f4a340d0ef0 2024-01-19T04:57:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.414+0000 7f4a31ffb640 1 -- 172.21.15.31:0/1909977988 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4a2804d440 con 0x7f4a340d0ef0 2024-01-19T04:57:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.415+0000 7f4a31ffb640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4a2006aed0 0x7f4a2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.415+0000 7f4a31ffb640 1 -- 172.21.15.31:0/1909977988 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4a280ab390 con 0x7f4a340d0ef0 2024-01-19T04:57:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.415+0000 7f4a391e4640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4a2006aed0 0x7f4a2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.415+0000 7f4a391e4640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4a2006aed0 0x7f4a2006d390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009ac0 tx=0x7f4a2c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.419+0000 7f4a31ffb640 1 -- 172.21.15.31:0/1909977988 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4a280765f0 con 0x7f4a340d0ef0 2024-01-19T04:57:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.596+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a340696e0 con 0x7f4a2006aed0 2024-01-19T04:57:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.599+0000 7f4a31ffb640 1 -- 172.21.15.31:0/1909977988 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51235 (secure 0 0 0) 0x7f4a340696e0 con 0x7f4a2006aed0 2024-01-19T04:57:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:49.600 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:57:49.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4a2006aed0 msgr2=0x7f4a2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4a2006aed0 0x7f4a2006d390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009ac0 tx=0x7f4a2c008040 comp rx=0 tx=0).stop 2024-01-19T04:57:49.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a340d0ef0 msgr2=0x7f4a340c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a340d0ef0 0x7f4a340c94c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034040 tx=0x7f4a280046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 shutdown_connections 2024-01-19T04:57:49.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4a2006aed0 0x7f4a2006d390 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a34160bb0 0x7f4a340c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a340d0ef0 0x7f4a340c94c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 --2- 172.21.15.31:0/1909977988 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4a340cfd10 0x7f4a341f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 >> 172.21.15.31:0/1909977988 conn(0x7f4a3409c600 msgr2=0x7f4a340b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:49.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 shutdown_connections 2024-01-19T04:57:49.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.601+0000 7f4a3b448640 1 -- 172.21.15.31:0/1909977988 wait complete. 2024-01-19T04:57:49.611 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-01-19T04:57:49.596698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222184,"kb_used_data":6016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752536,"statfs":{"total":773094113280,"available":772866596864,"internally_reserved":0,"allocated":6160384,"data_stored":3249833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001260"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-01-19T04:57:44.004901+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:57:44.004901+0000","last_peered":"2024-01-19T04:57:44.004901+0000","last_clean":"2024-01-19T04:57:44.004901+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:57:44.004901+0000","last_undegraded":"2024-01-19T04:57:44.004901+0000","last_fullsized":"2024-01-19T04:57:44.004901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":849674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73099999999999998,"5min":0.753,"15min":0.753},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68200000000000005,"5min":0.745,"15min":0.745},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68500000000000005,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":253955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.312,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.69999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.59899999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68799999999999994,"5min":0.876,"15min":0.876},"last":0.496}]},{"osd":3,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.69499999999999995,"5min":0.754,"15min":0.754},"last":0.435}]},{"osd":4,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.877,"15min":0.877},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.372}]},{"osd":7,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92800000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.504,"15min":0.505},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.496},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.624},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.51300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98299999999999998,"5min":1.103,"15min":1.103},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84799999999999998,"5min":1.2,"15min":1.2},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0049999999999999,"5min":1.321,"15min":1.321},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":1.046,"15min":1.046},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.48499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0309999999999999,"5min":1.258,"15min":1.258},"last":1.052}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.622}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.442},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.498}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.318,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53200000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.88,"15min":0.88},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.755,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80300000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71299999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81999999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67700000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.248},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":1.47,"15min":1.47},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":1.006,"15min":1.006},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90900000000000003,"5min":1.355,"15min":1.355},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82999999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81899999999999995,"5min":0.995,"15min":0.995},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":259306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56299999999999994,"5min":1.397,"15min":1.397},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.23799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":1.423,"15min":1.423},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66700000000000004,"5min":1.446,"15min":1.446},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.432},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":1.619,"15min":1.619},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":1.585,"15min":1.585},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":1.333,"15min":1.333},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83899999999999997,"5min":1.367,"15min":1.367},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.624},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":849674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.36899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46400000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38200000000000001}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.42299999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81200000000000006,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.044,"15min":1.044},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:57:49.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-19T04:57:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.714+0000 7f5d63858640 1 Processor -- start 2024-01-19T04:57:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.714+0000 7f5d63858640 1 -- start start 2024-01-19T04:57:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d63858640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d5c160a10 0x7f5d5c0b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d63858640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5d5c0d2e50 0x7f5d5c0d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d63858640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0b4d00 0x7f5d5c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d63858640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c160df0 con 0x7f5d5c160a10 2024-01-19T04:57:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d63858640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c0b5690 con 0x7f5d5c0d2e50 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d63858640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0d1ae0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d615f4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d5c160a10 0x7f5d5c0b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d615f4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d5c160a10 0x7f5d5c0b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58012/0 (socket says 172.21.15.31:58012) 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.715+0000 7f5d615f4640 1 -- 172.21.15.31:0/911437560 learned_addr learned my addr 172.21.15.31:0/911437560 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d60df3640 1 --2- 172.21.15.31:0/911437560 >> v2:172.21.15.137:3300/0 conn(0x7f5d5c0d2e50 0x7f5d5c0d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d61df5640 1 --2- 172.21.15.31:0/911437560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0b4d00 0x7f5d5c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d615f4640 1 -- 172.21.15.31:0/911437560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0b4d00 msgr2=0x7f5d5c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d615f4640 1 --2- 172.21.15.31:0/911437560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0b4d00 0x7f5d5c0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d615f4640 1 -- 172.21.15.31:0/911437560 >> v2:172.21.15.137:3300/0 conn(0x7f5d5c0d2e50 msgr2=0x7f5d5c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d615f4640 1 --2- 172.21.15.31:0/911437560 >> v2:172.21.15.137:3300/0 conn(0x7f5d5c0d2e50 0x7f5d5c0d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d615f4640 1 -- 172.21.15.31:0/911437560 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0d1db0 con 0x7f5d5c160a10 2024-01-19T04:57:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d60df3640 1 --2- 172.21.15.31:0/911437560 >> v2:172.21.15.137:3300/0 conn(0x7f5d5c0d2e50 0x7f5d5c0d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.716+0000 7f5d615f4640 1 --2- 172.21.15.31:0/911437560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c160a10 0x7f5d5c0b8ea0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0b6240 tx=0x7f5d5802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3255c634737688 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d53fff640 1 -- 172.21.15.31:0/911437560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58030860 con 0x7f5d5c160a10 2024-01-19T04:57:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d53fff640 1 -- 172.21.15.31:0/911437560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d580309c0 con 0x7f5d5c160a10 2024-01-19T04:57:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d53fff640 1 -- 172.21.15.31:0/911437560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58030d50 con 0x7f5d5c160a10 2024-01-19T04:57:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 -- 172.21.15.31:0/911437560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c160a10 msgr2=0x7f5d5c0b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 --2- 172.21.15.31:0/911437560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c160a10 0x7f5d5c0b8ea0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0b6240 tx=0x7f5d5802b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 -- 172.21.15.31:0/911437560 shutdown_connections 2024-01-19T04:57:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 --2- 172.21.15.31:0/911437560 >> v2:172.21.15.137:3300/0 conn(0x7f5d5c0d2e50 0x7f5d5c0d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 --2- 172.21.15.31:0/911437560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0b4d00 0x7f5d5c0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 --2- 172.21.15.31:0/911437560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c160a10 0x7f5d5c0b8ea0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 -- 172.21.15.31:0/911437560 >> 172.21.15.31:0/911437560 conn(0x7f5d5c072310 msgr2=0x7f5d5c0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 -- 172.21.15.31:0/911437560 shutdown_connections 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.717+0000 7f5d63858640 1 -- 172.21.15.31:0/911437560 wait complete. 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 Processor -- start 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 -- start start 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0b4d00 0x7f5d5c1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0d2e50 0x7f5d5c1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d5c160a10 0x7f5d5c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1f73d0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c155b20 con 0x7f5d5c160a10 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d63858640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c155e00 con 0x7f5d5c0d2e50 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d615f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0b4d00 0x7f5d5c1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d615f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0b4d00 0x7f5d5c1f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58014/0 (socket says 172.21.15.31:58014) 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.719+0000 7f5d615f4640 1 -- 172.21.15.31:0/2972132466 learned_addr learned my addr 172.21.15.31:0/2972132466 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d60df3640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0d2e50 0x7f5d5c1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d61df5640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d5c160a10 0x7f5d5c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d615f4640 1 -- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0d2e50 msgr2=0x7f5d5c1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d615f4640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0d2e50 0x7f5d5c1547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d615f4640 1 -- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d5c160a10 msgr2=0x7f5d5c1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d615f4640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d5c160a10 0x7f5d5c1577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d615f4640 1 -- 172.21.15.31:0/2972132466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d580094d0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d61df5640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d5c160a10 0x7f5d5c1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d60df3640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0d2e50 0x7f5d5c1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d615f4640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0b4d00 0x7f5d5c1f6e90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802b9e0 tx=0x7f5d58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d51ffb640 1 -- 172.21.15.31:0/2972132466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58039b60 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d51ffb640 1 -- 172.21.15.31:0/2972132466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58039ce0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1560e0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d51ffb640 1 -- 172.21.15.31:0/2972132466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58043650 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.720+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1566a0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.721+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c0696e0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.722+0000 7f5d51ffb640 1 -- 172.21.15.31:0/2972132466 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5d5804c3b0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.722+0000 7f5d51ffb640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5d4006aed0 0x7f5d4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.722+0000 7f5d51ffb640 1 -- 172.21.15.31:0/2972132466 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5d580ab7a0 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.722+0000 7f5d60df3640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5d4006aed0 0x7f5d4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.723+0000 7f5d60df3640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5d4006aed0 0x7f5d4006d390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0cd4c0 tx=0x7f5d4c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.726+0000 7f5d51ffb640 1 -- 172.21.15.31:0/2972132466 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5d58076970 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.905+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d5c067940 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.906+0000 7f5d51ffb640 1 -- 172.21.15.31:0/2972132466 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12951 (secure 0 0 0) 0x7f5d58034090 con 0x7f5d5c0b4d00 2024-01-19T04:57:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:49.906 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":53,"fsid":"16b8b5c3-25df-408f-9c77-cd32e346926d","created":"2024-01-19T04:54:54.736682+0000","modified":"2024-01-19T04:57:46.677657+0000","last_up_change":"2024-01-19T04:55:05.575070+0000","last_in_change":"2024-01-19T04:54:56.974222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-19T04:55:05.602959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51688160-057c-4b5a-951d-0a429534866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6810","nonce":2694559820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6814","nonce":2694559820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6822","nonce":2694559820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2694559820},{"type":"v1","addr":"172.21.15.31:6818","nonce":2694559820}]},"public_addr":"172.21.15.31:6810/2694559820","cluster_addr":"172.21.15.31:6814/2694559820","heartbeat_back_addr":"172.21.15.31:6822/2694559820","heartbeat_front_addr":"172.21.15.31:6818/2694559820","state":["exists","up"]},{"osd":1,"uuid":"c6763352-96f2-4658-b9b5-8e8c71d2a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6801","nonce":825319266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6803","nonce":825319266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6807","nonce":825319266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":825319266},{"type":"v1","addr":"172.21.15.31:6805","nonce":825319266}]},"public_addr":"172.21.15.31:6801/825319266","cluster_addr":"172.21.15.31:6803/825319266","heartbeat_back_addr":"172.21.15.31:6807/825319266","heartbeat_front_addr":"172.21.15.31:6805/825319266","state":["exists","up"]},{"osd":2,"uuid":"2e453883-d08d-4e0a-8e0e-cc6794eca4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6825","nonce":4257663237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6827","nonce":4257663237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6831","nonce":4257663237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4257663237},{"type":"v1","addr":"172.21.15.31:6829","nonce":4257663237}]},"public_addr":"172.21.15.31:6825/4257663237","cluster_addr":"172.21.15.31:6827/4257663237","heartbeat_back_addr":"172.21.15.31:6831/4257663237","heartbeat_front_addr":"172.21.15.31:6829/4257663237","state":["exists","up"]},{"osd":3,"uuid":"fc395e3e-0c30-4a22-8af8-8621c00739ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6809","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6811","nonce":3034750361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6813","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6815","nonce":3034750361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6823","nonce":3034750361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":3034750361},{"type":"v1","addr":"172.21.15.31:6819","nonce":3034750361}]},"public_addr":"172.21.15.31:6811/3034750361","cluster_addr":"172.21.15.31:6815/3034750361","heartbeat_back_addr":"172.21.15.31:6823/3034750361","heartbeat_front_addr":"172.21.15.31:6819/3034750361","state":["exists","up"]},{"osd":4,"uuid":"60ac628d-021f-40e5-8a3b-25b6911fa5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6817","nonce":7950289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6823","nonce":7950289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6831","nonce":7950289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":7950289},{"type":"v1","addr":"172.21.15.137:6828","nonce":7950289}]},"public_addr":"172.21.15.137:6817/7950289","cluster_addr":"172.21.15.137:6823/7950289","heartbeat_back_addr":"172.21.15.137:6831/7950289","heartbeat_front_addr":"172.21.15.137:6828/7950289","state":["exists","up"]},{"osd":5,"uuid":"3d85949f-f4e2-498a-ab2f-f7ba5c6b275a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6811","nonce":4102773142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6816","nonce":4102773142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6829","nonce":4102773142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":4102773142},{"type":"v1","addr":"172.21.15.137:6822","nonce":4102773142}]},"public_addr":"172.21.15.137:6811/4102773142","cluster_addr":"172.21.15.137:6816/4102773142","heartbeat_back_addr":"172.21.15.137:6829/4102773142","heartbeat_front_addr":"172.21.15.137:6822/4102773142","state":["exists","up"]},{"osd":6,"uuid":"f5b3d372-3610-4a0c-b90b-fbd5e205cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6810","nonce":4246993982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6815","nonce":4246993982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6827","nonce":4246993982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":4246993982},{"type":"v1","addr":"172.21.15.137:6821","nonce":4246993982}]},"public_addr":"172.21.15.137:6810/4246993982","cluster_addr":"172.21.15.137:6815/4246993982","heartbeat_back_addr":"172.21.15.137:6827/4246993982","heartbeat_front_addr":"172.21.15.137:6821/4246993982","state":["exists","up"]},{"osd":7,"uuid":"ce9ec718-829b-4a4e-af88-86ab19fc4e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6801","nonce":1281133662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6803","nonce":1281133662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6807","nonce":1281133662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1281133662},{"type":"v1","addr":"172.21.15.137:6805","nonce":1281133662}]},"public_addr":"172.21.15.137:6801/1281133662","cluster_addr":"172.21.15.137:6803/1281133662","heartbeat_back_addr":"172.21.15.137:6807/1281133662","heartbeat_front_addr":"172.21.15.137:6805/1281133662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.156001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.050263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.115929+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.138282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.220045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.209782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.204929+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-19T04:55:02.165489+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/1600502506":"2024-01-20T04:57:43.673535+0000","172.21.15.31:6834/1600502506":"2024-01-20T04:57:43.673535+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-19T04:57:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5d4006aed0 msgr2=0x7f5d4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5d4006aed0 0x7f5d4006d390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0cd4c0 tx=0x7f5d4c008040 comp rx=0 tx=0).stop 2024-01-19T04:57:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0b4d00 msgr2=0x7f5d5c1f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0b4d00 0x7f5d5c1f6e90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802b9e0 tx=0x7f5d58004730 comp rx=0 tx=0).stop 2024-01-19T04:57:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 shutdown_connections 2024-01-19T04:57:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5d4006aed0 0x7f5d4006d390 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d5c160a10 0x7f5d5c1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d5c0d2e50 0x7f5d5c1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 --2- 172.21.15.31:0/2972132466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0b4d00 0x7f5d5c1f6e90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:49.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 >> 172.21.15.31:0/2972132466 conn(0x7f5d5c072310 msgr2=0x7f5d5c0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:49.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 shutdown_connections 2024-01-19T04:57:49.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:49.908+0000 7f5d63858640 1 -- 172.21.15.31:0/2972132466 wait complete. 2024-01-19T04:57:50.919 INFO:tasks.ceph:Scrubbing osd.0 2024-01-19T04:57:50.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.037+0000 7f3371c18640 1 Processor -- start 2024-01-19T04:57:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.037+0000 7f3371c18640 1 -- start start 2024-01-19T04:57:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.038+0000 7f3371c18640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f336c0d36d0 0x7f336c0d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.038+0000 7f3371c18640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c0cfd30 con 0x7f336c0d3250 2024-01-19T04:57:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.038+0000 7f3371c18640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c0d0010 con 0x7f336c15d640 2024-01-19T04:57:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.038+0000 7f3371c18640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c0d09b0 con 0x7f336c0d36d0 2024-01-19T04:57:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.038+0000 7f336bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f336c0d36d0 0x7f336c0d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.038+0000 7f336affd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f336c0d3250 0x7f336c0d1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38284/0 (socket says 172.21.15.31:38284) 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.038+0000 7f336affd640 1 -- 172.21.15.31:0/3349147588 learned_addr learned my addr 172.21.15.31:0/3349147588 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.039+0000 7f336bfff640 1 -- 172.21.15.31:0/3349147588 >> v1:172.21.15.137:6789/0 conn(0x7f336c15d640 legacy=0x7f336c0b8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.039+0000 7f336bfff640 1 -- 172.21.15.31:0/3349147588 >> v1:172.21.15.31:6789/0 conn(0x7f336c0d3250 legacy=0x7f336c0d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.039+0000 7f336a7fc640 1 -- 172.21.15.31:0/3349147588 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670672328 0 0) 0x7f336c0cfd30 con 0x7f336c0d3250 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.039+0000 7f336bfff640 1 -- 172.21.15.31:0/3349147588 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c0d02f0 con 0x7f336c0d36d0 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f336bfff640 1 --2- 172.21.15.31:0/3349147588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d36d0 0x7f336c0d1e30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3364009d90 tx=0x7f336402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac21eddc9e8954fe server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f336a7fc640 1 -- 172.21.15.31:0/3349147588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336402cd00 con 0x7f336c0d36d0 2024-01-19T04:57:51.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f336a7fc640 1 -- 172.21.15.31:0/3349147588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336402ce60 con 0x7f336c0d36d0 2024-01-19T04:57:51.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f336a7fc640 1 -- 172.21.15.31:0/3349147588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364034660 con 0x7f336c0d36d0 2024-01-19T04:57:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f3371c18640 1 -- 172.21.15.31:0/3349147588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d36d0 msgr2=0x7f336c0d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f3371c18640 1 --2- 172.21.15.31:0/3349147588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d36d0 0x7f336c0d1e30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3364009d90 tx=0x7f336402bf50 comp rx=0 tx=0).stop 2024-01-19T04:57:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f3371c18640 1 -- 172.21.15.31:0/3349147588 shutdown_connections 2024-01-19T04:57:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f3371c18640 1 --2- 172.21.15.31:0/3349147588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d36d0 0x7f336c0d1e30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f3371c18640 1 -- 172.21.15.31:0/3349147588 >> 172.21.15.31:0/3349147588 conn(0x7f336c072310 msgr2=0x7f336c06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f3371c18640 1 -- 172.21.15.31:0/3349147588 shutdown_connections 2024-01-19T04:57:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.040+0000 7f3371c18640 1 -- 172.21.15.31:0/3349147588 wait complete. 2024-01-19T04:57:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.042+0000 7f3371c18640 1 Processor -- start 2024-01-19T04:57:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.042+0000 7f3371c18640 1 -- start start 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.042+0000 7f3371c18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d3250 0x7f336c151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3371c18640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f336c0d36d0 0x7f336c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3371c18640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f336c15d640 0x7f336c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3371c18640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c7ea0 con 0x7f336c0d3250 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3371c18640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0a9e40 con 0x7f336c15d640 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3371c18640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0aa120 con 0x7f336c0d36d0 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d3250 0x7f336c151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d3250 0x7f336c151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58036/0 (socket says 172.21.15.31:58036) 2024-01-19T04:57:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 -- 172.21.15.31:0/2142758876 learned_addr learned my addr 172.21.15.31:0/2142758876 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336bfff640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f336c15d640 0x7f336c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336affd640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f336c0d36d0 0x7f336c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 -- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f336c0d36d0 msgr2=0x7f336c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f336c0d36d0 0x7f336c0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 -- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f336c15d640 msgr2=0x7f336c14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f336c15d640 0x7f336c14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 -- 172.21.15.31:0/2142758876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364009a70 con 0x7f336c0d3250 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f336b7fe640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d3250 0x7f336c151e10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c400 tx=0x7f335c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c0052d0 con 0x7f336c0d3250 2024-01-19T04:57:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c005430 con 0x7f336c0d3250 2024-01-19T04:57:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c003830 con 0x7f336c0d3250 2024-01-19T04:57:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c0c80f0 con 0x7f336c0d3250 2024-01-19T04:57:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.043+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c0c8e30 con 0x7f336c0d3250 2024-01-19T04:57:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.044+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f336c0d3b20 con 0x7f336c0d3250 2024-01-19T04:57:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.045+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f335c003990 con 0x7f336c0d3250 2024-01-19T04:57:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.045+0000 7f3353fff640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f335806aed0 0x7f335806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.045+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f335c088750 con 0x7f336c0d3250 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.045+0000 7f3353fff640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f3358070d30 0x7f3358073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.045+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 --> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3358073800 con 0x7f3358070d30 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.045+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335c08ae10 con 0x7f336c0d3250 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.045+0000 7f336affd640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f335806aed0 0x7f335806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.046+0000 7f336bfff640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f3358070d30 0x7f3358073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.046+0000 7f336bfff640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f3358070d30 0x7f3358073150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.046+0000 7f336affd640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f335806aed0 0x7f335806d390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3360007f70 tx=0x7f3360007f00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.047+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== osd.0 v2:172.21.15.31:6808/2694559820 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3358073800 con 0x7f3358070d30 2024-01-19T04:57:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 --> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f336c0c8610 con 0x7f3358070d30 2024-01-19T04:57:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3353fff640 1 -- 172.21.15.31:0/2142758876 <== osd.0 v2:172.21.15.31:6808/2694559820 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f336c0c8610 con 0x7f3358070d30 2024-01-19T04:57:51.060 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:51.060 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:51.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f3358070d30 msgr2=0x7f3358073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:6808/2694559820,v1:172.21.15.31:6810/2694559820] conn(0x7f3358070d30 0x7f3358073150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f335806aed0 msgr2=0x7f335806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f335806aed0 0x7f335806d390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3360007f70 tx=0x7f3360007f00 comp rx=0 tx=0).stop 2024-01-19T04:57:51.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d3250 msgr2=0x7f336c151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 --2- 172.21.15.31:0/2142758876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0d3250 0x7f336c151e10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c400 tx=0x7f335c00c8c0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f336bfff640 1 -- 172.21.15.31:0/2142758876 reap_dead start 2024-01-19T04:57:51.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 shutdown_connections 2024-01-19T04:57:51.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.060+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 >> 172.21.15.31:0/2142758876 conn(0x7f336c072310 msgr2=0x7f336c0b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.061+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 shutdown_connections 2024-01-19T04:57:51.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.061+0000 7f3371c18640 1 -- 172.21.15.31:0/2142758876 wait complete. 2024-01-19T04:57:51.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-19T04:57:51.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.173+0000 7f223f3b0640 1 Processor -- start 2024-01-19T04:57:51.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223f3b0640 1 -- start start 2024-01-19T04:57:51.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223f3b0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22380cf6f0 0x7f22380add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223f3b0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f22380b1d60 0x7f22380b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223f3b0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22380b0a70 con 0x7f22380cfad0 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223f3b0640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22380b18d0 con 0x7f22380cf6f0 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223f3b0640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22380b0cc0 con 0x7f22380b1d60 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223d14c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22380cf6f0 0x7f22380add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223c94b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f22380cfad0 0x7f22380cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38294/0 (socket says 172.21.15.31:38294) 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.174+0000 7f223c94b640 1 -- 172.21.15.31:0/3262205235 learned_addr learned my addr 172.21.15.31:0/3262205235 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.175+0000 7f223d94d640 1 --2- 172.21.15.31:0/3262205235 >> v2:172.21.15.137:3300/0 conn(0x7f22380b1d60 0x7f22380b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.175+0000 7f223d14c640 1 -- 172.21.15.31:0/3262205235 >> v2:172.21.15.137:3300/0 conn(0x7f22380b1d60 msgr2=0x7f22380b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:51.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.175+0000 7f223d14c640 1 --2- 172.21.15.31:0/3262205235 >> v2:172.21.15.137:3300/0 conn(0x7f22380b1d60 0x7f22380b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.175+0000 7f223d14c640 1 -- 172.21.15.31:0/3262205235 >> v1:172.21.15.31:6789/0 conn(0x7f22380cfad0 legacy=0x7f22380cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.175+0000 7f223d14c640 1 -- 172.21.15.31:0/3262205235 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380b0fa0 con 0x7f22380cf6f0 2024-01-19T04:57:51.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.175+0000 7f222bfff640 1 -- 172.21.15.31:0/3262205235 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487681426 0 0) 0x7f22380b0a70 con 0x7f22380cfad0 2024-01-19T04:57:51.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.175+0000 7f223d14c640 1 --2- 172.21.15.31:0/3262205235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380cf6f0 0x7f22380add00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22380c8170 tx=0x7f222c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a00fccba5a5e3b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f222bfff640 1 -- 172.21.15.31:0/3262205235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c031830 con 0x7f22380cf6f0 2024-01-19T04:57:51.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f222bfff640 1 -- 172.21.15.31:0/3262205235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c031990 con 0x7f22380cf6f0 2024-01-19T04:57:51.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f222bfff640 1 -- 172.21.15.31:0/3262205235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c031d20 con 0x7f22380cf6f0 2024-01-19T04:57:51.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 -- 172.21.15.31:0/3262205235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380cf6f0 msgr2=0x7f22380add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 --2- 172.21.15.31:0/3262205235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380cf6f0 0x7f22380add00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22380c8170 tx=0x7f222c02b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 -- 172.21.15.31:0/3262205235 shutdown_connections 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 --2- 172.21.15.31:0/3262205235 >> v2:172.21.15.137:3300/0 conn(0x7f22380b1d60 0x7f22380b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 --2- 172.21.15.31:0/3262205235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380cf6f0 0x7f22380add00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 -- 172.21.15.31:0/3262205235 >> 172.21.15.31:0/3262205235 conn(0x7f223808afe0 msgr2=0x7f22380d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 -- 172.21.15.31:0/3262205235 shutdown_connections 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.176+0000 7f223f3b0640 1 -- 172.21.15.31:0/3262205235 wait complete. 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.177+0000 7f223f3b0640 1 Processor -- start 2024-01-19T04:57:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223f3b0640 1 -- start start 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223f3b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380b1d60 0x7f22380c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223f3b0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22380cf6f0 0x7f2238151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223f3b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22380cfad0 0x7f2238159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223f3b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2238158ab0 con 0x7f22380b1d60 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223f3b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2238151a80 con 0x7f22380cfad0 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223f3b0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223806d6b0 con 0x7f22380cf6f0 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223d14c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380b1d60 0x7f22380c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223d14c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380b1d60 0x7f22380c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58058/0 (socket says 172.21.15.31:58058) 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223d14c640 1 -- 172.21.15.31:0/268150167 learned_addr learned my addr 172.21.15.31:0/268150167 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.178+0000 7f223d94d640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22380cfad0 0x7f2238159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223c94b640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22380cf6f0 0x7f2238151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223d14c640 1 -- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22380cf6f0 msgr2=0x7f2238151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223d14c640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22380cf6f0 0x7f2238151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223d14c640 1 -- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22380cfad0 msgr2=0x7f2238159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223d14c640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22380cfad0 0x7f2238159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223d14c640 1 -- 172.21.15.31:0/268150167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c009400 con 0x7f22380b1d60 2024-01-19T04:57:51.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223c94b640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22380cf6f0 0x7f2238151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223d14c640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380b1d60 0x7f22380c9040 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f222c000c00 tx=0x7f222c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f2229ffb640 1 -- 172.21.15.31:0/268150167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c004230 con 0x7f22380b1d60 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f2229ffb640 1 -- 172.21.15.31:0/268150167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c0043b0 con 0x7f22380b1d60 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22380c8280 con 0x7f22380b1d60 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f2229ffb640 1 -- 172.21.15.31:0/268150167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c004520 con 0x7f22380b1d60 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.179+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22380c8840 con 0x7f22380b1d60 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.180+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2238074900 con 0x7f22380b1d60 2024-01-19T04:57:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.181+0000 7f2229ffb640 1 -- 172.21.15.31:0/268150167 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f222c03a8a0 con 0x7f22380b1d60 2024-01-19T04:57:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.181+0000 7f2229ffb640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f221c06aed0 0x7f221c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.181+0000 7f2229ffb640 1 -- 172.21.15.31:0/268150167 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f222c0abb10 con 0x7f22380b1d60 2024-01-19T04:57:51.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.181+0000 7f223c94b640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f221c06aed0 0x7f221c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.182+0000 7f223c94b640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f221c06aed0 0x7f221c06d390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f22380afb50 tx=0x7f2234007c30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.185+0000 7f2229ffb640 1 -- 172.21.15.31:0/268150167 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f222c076d70 con 0x7f22380b1d60 2024-01-19T04:57:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.366+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f22380b1560 con 0x7f221c06aed0 2024-01-19T04:57:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.366+0000 7f2229ffb640 1 -- 172.21.15.31:0/268150167 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2234005c80 con 0x7f221c06aed0 2024-01-19T04:57:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2024-01-19T04:57:51.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.368+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f221c06aed0 msgr2=0x7f221c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.368+0000 7f223f3b0640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f221c06aed0 0x7f221c06d390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f22380afb50 tx=0x7f2234007c30 comp rx=0 tx=0).stop 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.368+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380b1d60 msgr2=0x7f22380c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.368+0000 7f223f3b0640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380b1d60 0x7f22380c9040 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f222c000c00 tx=0x7f222c0046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 shutdown_connections 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f221c06aed0 0x7f221c06d390 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22380cfad0 0x7f2238159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f22380cf6f0 0x7f2238151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 --2- 172.21.15.31:0/268150167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22380b1d60 0x7f22380c9040 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 >> 172.21.15.31:0/268150167 conn(0x7f223808afe0 msgr2=0x7f22380cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 shutdown_connections 2024-01-19T04:57:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.369+0000 7f223f3b0640 1 -- 172.21.15.31:0/268150167 wait complete. 2024-01-19T04:57:51.379 INFO:tasks.ceph:Scrubbing osd.1 2024-01-19T04:57:51.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.485+0000 7ffa01aaf640 1 Processor -- start 2024-01-19T04:57:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.485+0000 7ffa01aaf640 1 -- start start 2024-01-19T04:57:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.486+0000 7ffa01aaf640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc15d640 0x7ff9fc0b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.486+0000 7ffa01aaf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc161e90 con 0x7ff9fc0d2730 2024-01-19T04:57:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.486+0000 7ffa01aaf640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc0cfd30 con 0x7ff9fc0d3250 2024-01-19T04:57:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.486+0000 7ffa01aaf640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0d09b0 con 0x7ff9fc15d640 2024-01-19T04:57:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.486+0000 7ff9fbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff9fc0d2730 0x7ff9fc161780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38300/0 (socket says 172.21.15.31:38300) 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.486+0000 7ff9fbfff640 1 -- 172.21.15.31:0/4241259341 learned_addr learned my addr 172.21.15.31:0/4241259341 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.486+0000 7ff9fb7fe640 1 --2- 172.21.15.31:0/4241259341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc15d640 0x7ff9fc0b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.487+0000 7ff9fa7fc640 1 -- 172.21.15.31:0/4241259341 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714412515 0 0) 0x7ff9fc161e90 con 0x7ff9fc0d2730 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.487+0000 7ff9fa7fc640 1 -- 172.21.15.31:0/4241259341 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e4003520 con 0x7ff9fc0d2730 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.487+0000 7ff9fa7fc640 1 -- 172.21.15.31:0/4241259341 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945914379 0 0) 0x7ff9fc0cfd30 con 0x7ff9fc0d3250 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.487+0000 7ff9fa7fc640 1 -- 172.21.15.31:0/4241259341 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9fc161e90 con 0x7ff9fc0d3250 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.487+0000 7ff9fb7fe640 1 -- 172.21.15.31:0/4241259341 >> v1:172.21.15.137:6789/0 conn(0x7ff9fc0d3250 legacy=0x7ff9fc161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.487+0000 7ff9fb7fe640 1 -- 172.21.15.31:0/4241259341 >> v1:172.21.15.31:6789/0 conn(0x7ff9fc0d2730 legacy=0x7ff9fc161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.487+0000 7ff9fb7fe640 1 -- 172.21.15.31:0/4241259341 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc0d0010 con 0x7ff9fc15d640 2024-01-19T04:57:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ff9fb7fe640 1 --2- 172.21.15.31:0/4241259341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc15d640 0x7ff9fc0b8be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0097d0 tx=0x7ff9ec02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be2ebdd0e4c9fea server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ff9fa7fc640 1 -- 172.21.15.31:0/4241259341 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec031890 con 0x7ff9fc15d640 2024-01-19T04:57:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ff9fa7fc640 1 -- 172.21.15.31:0/4241259341 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec0319f0 con 0x7ff9fc15d640 2024-01-19T04:57:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ff9fa7fc640 1 -- 172.21.15.31:0/4241259341 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec031d60 con 0x7ff9fc15d640 2024-01-19T04:57:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ffa01aaf640 1 -- 172.21.15.31:0/4241259341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc15d640 msgr2=0x7ff9fc0b8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ffa01aaf640 1 --2- 172.21.15.31:0/4241259341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc15d640 0x7ff9fc0b8be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0097d0 tx=0x7ff9ec02b930 comp rx=0 tx=0).stop 2024-01-19T04:57:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ffa01aaf640 1 -- 172.21.15.31:0/4241259341 shutdown_connections 2024-01-19T04:57:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ffa01aaf640 1 --2- 172.21.15.31:0/4241259341 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc15d640 0x7ff9fc0b8be0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ffa01aaf640 1 -- 172.21.15.31:0/4241259341 >> 172.21.15.31:0/4241259341 conn(0x7ff9fc072310 msgr2=0x7ff9fc06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ffa01aaf640 1 -- 172.21.15.31:0/4241259341 shutdown_connections 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.488+0000 7ffa01aaf640 1 -- 172.21.15.31:0/4241259341 wait complete. 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.489+0000 7ffa01aaf640 1 Processor -- start 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ffa01aaf640 1 -- start start 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ffa01aaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9fc0d2730 0x7ff9fc151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ffa01aaf640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc0d3250 0x7ff9fc0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ffa01aaf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9fc15d640 0x7ff9fc1f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ffa01aaf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0ca390 con 0x7ff9fc15d640 2024-01-19T04:57:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ffa01aaf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1f3c60 con 0x7ff9fc0d2730 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ffa01aaf640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1f3f40 con 0x7ff9fc0d3250 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.490+0000 7ff9fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9fc15d640 0x7ff9fc1f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9fc15d640 0x7ff9fc1f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58072/0 (socket says 172.21.15.31:58072) 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 -- 172.21.15.31:0/1148730820 learned_addr learned my addr 172.21.15.31:0/1148730820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fb7fe640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9fc0d2730 0x7ff9fc151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9faffd640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc0d3250 0x7ff9fc0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 -- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc0d3250 msgr2=0x7ff9fc0cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff9fc0d3250 0x7ff9fc0cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 -- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9fc0d2730 msgr2=0x7ff9fc151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9fc0d2730 0x7ff9fc151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 -- 172.21.15.31:0/1148730820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec009400 con 0x7ff9fc15d640 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9fbfff640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9fc15d640 0x7ff9fc1f3120 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cb10 tx=0x7ff9f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400ac70 con 0x7ff9fc15d640 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f400adf0 con 0x7ff9fc15d640 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc0ca5a0 con 0x7ff9fc15d640 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4012640 con 0x7ff9fc15d640 2024-01-19T04:57:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.491+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc151c80 con 0x7ff9fc15d640 2024-01-19T04:57:51.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.492+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9fc0d05d0 con 0x7ff9fc15d640 2024-01-19T04:57:51.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff9f4007070 con 0x7ff9fc15d640 2024-01-19T04:57:51.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9e3fff640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff9e806aed0 0x7ff9e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff9f4089210 con 0x7ff9fc15d640 2024-01-19T04:57:51.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9e3fff640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7ff9e8070d30 0x7ff9e8073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 --> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9e8073800 con 0x7ff9e8070d30 2024-01-19T04:57:51.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f408b870 con 0x7ff9fc15d640 2024-01-19T04:57:51.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9fb7fe640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff9e806aed0 0x7ff9e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.493+0000 7ff9faffd640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7ff9e8070d30 0x7ff9e8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.494+0000 7ff9fb7fe640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff9e806aed0 0x7ff9e806d390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.494+0000 7ff9faffd640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7ff9e8070d30 0x7ff9e8073150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.494+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== osd.1 v2:172.21.15.31:6800/825319266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff9e8073800 con 0x7ff9e8070d30 2024-01-19T04:57:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.508+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 --> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff9fc0caac0 con 0x7ff9e8070d30 2024-01-19T04:57:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.508+0000 7ff9e3fff640 1 -- 172.21.15.31:0/1148730820 <== osd.1 v2:172.21.15.31:6800/825319266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff9fc0caac0 con 0x7ff9e8070d30 2024-01-19T04:57:51.508 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:51.508 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:51.508 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.508+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7ff9e8070d30 msgr2=0x7ff9e8073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.508+0000 7ffa01aaf640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:6800/825319266,v1:172.21.15.31:6801/825319266] conn(0x7ff9e8070d30 0x7ff9e8073150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff9e806aed0 msgr2=0x7ff9e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff9e806aed0 0x7ff9e806d390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec03c040 comp rx=0 tx=0).stop 2024-01-19T04:57:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9fc15d640 msgr2=0x7ff9fc1f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 --2- 172.21.15.31:0/1148730820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9fc15d640 0x7ff9fc1f3120 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cb10 tx=0x7ff9f400a060 comp rx=0 tx=0).stop 2024-01-19T04:57:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ff9fbfff640 1 -- 172.21.15.31:0/1148730820 reap_dead start 2024-01-19T04:57:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 shutdown_connections 2024-01-19T04:57:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 >> 172.21.15.31:0/1148730820 conn(0x7ff9fc072310 msgr2=0x7ff9fc15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 shutdown_connections 2024-01-19T04:57:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.509+0000 7ffa01aaf640 1 -- 172.21.15.31:0/1148730820 wait complete. 2024-01-19T04:57:51.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-19T04:57:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.621+0000 7f9c420da640 1 Processor -- start 2024-01-19T04:57:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.621+0000 7f9c420da640 1 -- start start 2024-01-19T04:57:51.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c420da640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9c3c0d3f10 0x7f9c3c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c420da640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c0cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c420da640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c0d1450 con 0x7f9c3c15d5c0 2024-01-19T04:57:51.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c420da640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c0d22d0 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c420da640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cf240 con 0x7f9c3c0d1e40 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c3b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9c3c15d5c0 0x7f9c3c0cdec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38302/0 (socket says 172.21.15.31:38302) 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c3b7fe640 1 -- 172.21.15.31:0/485179945 learned_addr learned my addr 172.21.15.31:0/485179945 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c3affd640 1 --2- 172.21.15.31:0/485179945 >> v2:172.21.15.137:3300/0 conn(0x7f9c3c0d3f10 0x7f9c3c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.622+0000 7f9c3bfff640 1 --2- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c0cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.623+0000 7f9c3a7fc640 1 -- 172.21.15.31:0/485179945 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714020560 0 0) 0x7f9c3c0d1450 con 0x7f9c3c15d5c0 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.623+0000 7f9c3a7fc640 1 -- 172.21.15.31:0/485179945 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c24003520 con 0x7f9c3c15d5c0 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.623+0000 7f9c3affd640 1 -- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 msgr2=0x7f9c3c0cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.623+0000 7f9c3affd640 1 --2- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c0cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.623+0000 7f9c3affd640 1 -- 172.21.15.31:0/485179945 >> v1:172.21.15.31:6789/0 conn(0x7f9c3c15d5c0 legacy=0x7f9c3c0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.623+0000 7f9c3affd640 1 -- 172.21.15.31:0/485179945 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0ce6c0 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c3affd640 1 --2- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0d1900 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c300097a0 tx=0x7f9c3002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e1b4ce7cc74baf server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c3a7fc640 1 -- 172.21.15.31:0/485179945 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30031830 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c3a7fc640 1 -- 172.21.15.31:0/485179945 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c30031990 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c3a7fc640 1 -- 172.21.15.31:0/485179945 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30031d20 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 -- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c0d3f10 msgr2=0x7f9c3c0d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 --2- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0d1900 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c300097a0 tx=0x7f9c3002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 -- 172.21.15.31:0/485179945 shutdown_connections 2024-01-19T04:57:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 --2- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c0cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 --2- 172.21.15.31:0/485179945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0d1900 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 -- 172.21.15.31:0/485179945 >> 172.21.15.31:0/485179945 conn(0x7f9c3c09c600 msgr2=0x7f9c3c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 -- 172.21.15.31:0/485179945 shutdown_connections 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.624+0000 7f9c420da640 1 -- 172.21.15.31:0/485179945 wait complete. 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.625+0000 7f9c420da640 1 Processor -- start 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c420da640 1 -- start start 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c420da640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c420da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c420da640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c15d5c0 0x7f9c3c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c420da640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cc410 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c420da640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cc6f0 con 0x7f9c3c15d5c0 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c420da640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cc9d0 con 0x7f9c3c0d1e40 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c3affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c3affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58082/0 (socket says 172.21.15.31:58082) 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.626+0000 7f9c3affd640 1 -- 172.21.15.31:0/2379927886 learned_addr learned my addr 172.21.15.31:0/2379927886 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3b7fe640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3bfff640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c15d5c0 0x7f9c3c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3affd640 1 -- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 msgr2=0x7f9c3c15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3affd640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c15df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3affd640 1 -- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c15d5c0 msgr2=0x7f9c3c0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3affd640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c15d5c0 0x7f9c3c0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3affd640 1 -- 172.21.15.31:0/2379927886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30009400 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c3affd640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0cb8c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9c300098c0 tx=0x7f9c300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c23fff640 1 -- 172.21.15.31:0/2379927886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3003a630 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c23fff640 1 -- 172.21.15.31:0/2379927886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3003a7b0 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c23fff640 1 -- 172.21.15.31:0/2379927886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3003ab60 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0cccb0 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.627+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c1609c0 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.628+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c160b50 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.629+0000 7f9c23fff640 1 -- 172.21.15.31:0/2379927886 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f9c3003acc0 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.629+0000 7f9c23fff640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c2806aed0 0x7f9c2806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.629+0000 7f9c23fff640 1 -- 172.21.15.31:0/2379927886 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9c300aac20 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.629+0000 7f9c3b7fe640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c2806aed0 0x7f9c2806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.630+0000 7f9c3b7fe640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c2806aed0 0x7f9c2806d390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c005880 tx=0x7f9c2c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.633+0000 7f9c23fff640 1 -- 172.21.15.31:0/2379927886 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9c30075f40 con 0x7f9c3c0d3f10 2024-01-19T04:57:51.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.812+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c3c0cef60 con 0x7f9c2806aed0 2024-01-19T04:57:51.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.812+0000 7f9c23fff640 1 -- 172.21.15.31:0/2379927886 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9c2c001580 con 0x7f9c2806aed0 2024-01-19T04:57:51.812 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2024-01-19T04:57:51.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c2806aed0 msgr2=0x7f9c2806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c2806aed0 0x7f9c2806d390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c005880 tx=0x7f9c2c009040 comp rx=0 tx=0).stop 2024-01-19T04:57:51.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c3c0d3f10 msgr2=0x7f9c3c0cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0cb8c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9c300098c0 tx=0x7f9c300046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 shutdown_connections 2024-01-19T04:57:51.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f9c2806aed0 0x7f9c2806d390 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c3c15d5c0 0x7f9c3c0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c3c0d3f10 0x7f9c3c0cb8c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 --2- 172.21.15.31:0/2379927886 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c3c0d1e40 0x7f9c3c15df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 >> 172.21.15.31:0/2379927886 conn(0x7f9c3c09c600 msgr2=0x7f9c3c0b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 shutdown_connections 2024-01-19T04:57:51.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.814+0000 7f9c420da640 1 -- 172.21.15.31:0/2379927886 wait complete. 2024-01-19T04:57:51.824 INFO:tasks.ceph:Scrubbing osd.2 2024-01-19T04:57:51.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:51.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.930+0000 7f72f9d78640 1 Processor -- start 2024-01-19T04:57:51.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.930+0000 7f72f9d78640 1 -- start start 2024-01-19T04:57:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f9d78640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f72f4144a70 0x7f72f415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f9d78640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72f406acc0 0x7f72f415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f9d78640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f415f390 con 0x7f72f4071f20 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f9d78640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f415fe60 con 0x7f72f406acc0 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f9d78640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f406ebe0 con 0x7f72f4144a70 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72f406acc0 0x7f72f415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f2ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f72f4071f20 0x7f72f406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38318/0 (socket says 172.21.15.31:38318) 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f2ffd640 1 -- 172.21.15.31:0/1372588362 learned_addr learned my addr 172.21.15.31:0/1372588362 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f3fff640 1 -- 172.21.15.31:0/1372588362 >> v2:172.21.15.137:3300/0 conn(0x7f72f4144a70 msgr2=0x7f72f415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:51.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f37fe640 1 --2- 172.21.15.31:0/1372588362 >> v2:172.21.15.137:3300/0 conn(0x7f72f4144a70 0x7f72f415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f3fff640 1 --2- 172.21.15.31:0/1372588362 >> v2:172.21.15.137:3300/0 conn(0x7f72f4144a70 0x7f72f415ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f27fc640 1 -- 172.21.15.31:0/1372588362 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314435430 0 0) 0x7f72f415f390 con 0x7f72f4071f20 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f3fff640 1 -- 172.21.15.31:0/1372588362 >> v1:172.21.15.31:6789/0 conn(0x7f72f4071f20 legacy=0x7f72f406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.931+0000 7f72f3fff640 1 -- 172.21.15.31:0/1372588362 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40723d0 con 0x7f72f406acc0 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.932+0000 7f72f3fff640 1 --2- 172.21.15.31:0/1372588362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f415ee50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f72f406e630 tx=0x7f72ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0052cdd679c729 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.932+0000 7f72f27fc640 1 -- 172.21.15.31:0/1372588362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec02cbf0 con 0x7f72f406acc0 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.932+0000 7f72f27fc640 1 -- 172.21.15.31:0/1372588362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec02cd50 con 0x7f72f406acc0 2024-01-19T04:57:51.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.932+0000 7f72f27fc640 1 -- 172.21.15.31:0/1372588362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec034660 con 0x7f72f406acc0 2024-01-19T04:57:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.932+0000 7f72f9d78640 1 -- 172.21.15.31:0/1372588362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 msgr2=0x7f72f415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.932+0000 7f72f9d78640 1 --2- 172.21.15.31:0/1372588362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f415ee50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f72f406e630 tx=0x7f72ec02be40 comp rx=0 tx=0).stop 2024-01-19T04:57:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.933+0000 7f72f9d78640 1 -- 172.21.15.31:0/1372588362 shutdown_connections 2024-01-19T04:57:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.933+0000 7f72f9d78640 1 --2- 172.21.15.31:0/1372588362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f415ee50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.933+0000 7f72f9d78640 1 --2- 172.21.15.31:0/1372588362 >> v2:172.21.15.137:3300/0 conn(0x7f72f4144a70 0x7f72f415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.933+0000 7f72f9d78640 1 -- 172.21.15.31:0/1372588362 >> 172.21.15.31:0/1372588362 conn(0x7f72f406d750 msgr2=0x7f72f406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.933+0000 7f72f9d78640 1 -- 172.21.15.31:0/1372588362 shutdown_connections 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.933+0000 7f72f9d78640 1 -- 172.21.15.31:0/1372588362 wait complete. 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.934+0000 7f72f9d78640 1 Processor -- start 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.934+0000 7f72f9d78640 1 -- start start 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72f4071f20 0x7f72f415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72f4144a70 0x7f72f40b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f415b130 con 0x7f72f406acc0 2024-01-19T04:57:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f413d0d0 con 0x7f72f4071f20 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f413d390 con 0x7f72f4144a70 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f40cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58100/0 (socket says 172.21.15.31:58100) 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 -- 172.21.15.31:0/2525619160 learned_addr learned my addr 172.21.15.31:0/2525619160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f3fff640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72f4144a70 0x7f72f40b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f2ffd640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72f4071f20 0x7f72f415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 -- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72f4144a70 msgr2=0x7f72f40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72f4144a70 0x7f72f40b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 -- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72f4071f20 msgr2=0x7f72f415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72f4071f20 0x7f72f415abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 -- 172.21.15.31:0/2525619160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec009a70 con 0x7f72f406acc0 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f37fe640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f40cb930 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f72e400c7f0 tx=0x7f72e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4004220 con 0x7f72f406acc0 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e40043a0 con 0x7f72f406acc0 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f40b4240 con 0x7f72f406acc0 2024-01-19T04:57:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4007860 con 0x7f72f406acc0 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.935+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f40b39a0 con 0x7f72f406acc0 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.936+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72f415fb80 con 0x7f72f406acc0 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f72e4007bd0 con 0x7f72f406acc0 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72dbfff640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f72e006af00 0x7f72e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f72e40891b0 con 0x7f72f406acc0 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72f2ffd640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f72e006af00 0x7f72e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72dbfff640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f72e0070d60 0x7f72e0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 --> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72e0073830 con 0x7f72e0070d60 2024-01-19T04:57:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e408b870 con 0x7f72f406acc0 2024-01-19T04:57:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.937+0000 7f72f3fff640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f72e0070d60 0x7f72e0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.938+0000 7f72f2ffd640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f72e006af00 0x7f72e006d3c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ca670 tx=0x7f72e8007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.938+0000 7f72f3fff640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f72e0070d60 0x7f72e0073180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.938+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== osd.2 v2:172.21.15.31:6824/4257663237 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f72e0073830 con 0x7f72e0070d60 2024-01-19T04:57:51.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.951+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 --> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f72f40b3f30 con 0x7f72e0070d60 2024-01-19T04:57:51.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72dbfff640 1 -- 172.21.15.31:0/2525619160 <== osd.2 v2:172.21.15.31:6824/4257663237 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f72f40b3f30 con 0x7f72e0070d60 2024-01-19T04:57:51.951 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:51.952 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f72e0070d60 msgr2=0x7f72e0073180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:6824/4257663237,v1:172.21.15.31:6825/4257663237] conn(0x7f72e0070d60 0x7f72e0073180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f72e006af00 msgr2=0x7f72e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f72e006af00 0x7f72e006d3c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ca670 tx=0x7f72e8007d50 comp rx=0 tx=0).stop 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 msgr2=0x7f72f40cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 --2- 172.21.15.31:0/2525619160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72f406acc0 0x7f72f40cb930 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f72e400c7f0 tx=0x7f72e400ccb0 comp rx=0 tx=0).stop 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f3fff640 1 -- 172.21.15.31:0/2525619160 reap_dead start 2024-01-19T04:57:51.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 shutdown_connections 2024-01-19T04:57:51.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 >> 172.21.15.31:0/2525619160 conn(0x7f72f406d750 msgr2=0x7f72f40b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:51.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 shutdown_connections 2024-01-19T04:57:51.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:51.952+0000 7f72f9d78640 1 -- 172.21.15.31:0/2525619160 wait complete. 2024-01-19T04:57:51.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-19T04:57:52.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.064+0000 7f9006c08640 1 Processor -- start 2024-01-19T04:57:52.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.064+0000 7f9006c08640 1 -- start start 2024-01-19T04:57:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f9006c08640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9000161860 0x7f9000168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f9006c08640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90000cf9f0 con 0x7f900015d3a0 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f9006c08640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90000cfcd0 con 0x7f90000cf570 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f9006c08640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90000cffb0 con 0x7f9000161860 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f90051a5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9000161860 0x7f9000168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f90049a4640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f900015d3a0 0x7f900006c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38332/0 (socket says 172.21.15.31:38332) 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f90049a4640 1 -- 172.21.15.31:0/2067638495 learned_addr learned my addr 172.21.15.31:0/2067638495 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f90051a5640 1 -- 172.21.15.31:0/2067638495 >> v1:172.21.15.137:6789/0 conn(0x7f90000cf570 legacy=0x7f9000161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f90051a5640 1 -- 172.21.15.31:0/2067638495 >> v1:172.21.15.31:6789/0 conn(0x7f900015d3a0 legacy=0x7f900006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.065+0000 7f90051a5640 1 -- 172.21.15.31:0/2067638495 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000d0290 con 0x7f9000161860 2024-01-19T04:57:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.066+0000 7f90051a5640 1 --2- 172.21.15.31:0/2067638495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000168490 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00097a0 tx=0x7f8ff002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c159eb76e77f0a92 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.066+0000 7f8fff7fe640 1 -- 172.21.15.31:0/2067638495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff002e070 con 0x7f9000161860 2024-01-19T04:57:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.066+0000 7f8fff7fe640 1 -- 172.21.15.31:0/2067638495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff00315a0 con 0x7f9000161860 2024-01-19T04:57:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.066+0000 7f8fff7fe640 1 -- 172.21.15.31:0/2067638495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0031930 con 0x7f9000161860 2024-01-19T04:57:52.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.067+0000 7f9006c08640 1 -- 172.21.15.31:0/2067638495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 msgr2=0x7f9000168490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.067+0000 7f9006c08640 1 --2- 172.21.15.31:0/2067638495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000168490 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00097a0 tx=0x7f8ff002b840 comp rx=0 tx=0).stop 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.067+0000 7f9006c08640 1 -- 172.21.15.31:0/2067638495 shutdown_connections 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.067+0000 7f9006c08640 1 --2- 172.21.15.31:0/2067638495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000168490 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.067+0000 7f9006c08640 1 -- 172.21.15.31:0/2067638495 >> 172.21.15.31:0/2067638495 conn(0x7f90000d1100 msgr2=0x7f90000d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.067+0000 7f9006c08640 1 -- 172.21.15.31:0/2067638495 shutdown_connections 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.067+0000 7f9006c08640 1 -- 172.21.15.31:0/2067638495 wait complete. 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.068+0000 7f9006c08640 1 Processor -- start 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.068+0000 7f9006c08640 1 -- start start 2024-01-19T04:57:52.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f9006c08640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90000cf570 0x7f90001f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f9006c08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900015d3a0 0x7f90000c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f9006c08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f9006c08640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000b19c0 con 0x7f9000161860 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f9006c08640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001f3420 con 0x7f900015d3a0 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f9006c08640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90000b1400 con 0x7f90000cf570 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58128/0 (socket says 172.21.15.31:58128) 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 -- 172.21.15.31:0/1328979047 learned_addr learned my addr 172.21.15.31:0/1328979047 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90049a4640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90000cf570 0x7f90001f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f8ffffff640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900015d3a0 0x7f90000c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 -- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90000cf570 msgr2=0x7f90001f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90000cf570 0x7f90001f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 -- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900015d3a0 msgr2=0x7f90000c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900015d3a0 0x7f90000c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 -- 172.21.15.31:0/1328979047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0009400 con 0x7f9000161860 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.069+0000 7f90051a5640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000158be0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00097a0 tx=0x7f8ff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.070+0000 7f8ffd7fa640 1 -- 172.21.15.31:0/1328979047 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0032840 con 0x7f9000161860 2024-01-19T04:57:52.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.070+0000 7f8ffd7fa640 1 -- 172.21.15.31:0/1328979047 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff00329c0 con 0x7f9000161860 2024-01-19T04:57:52.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.070+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90000b16e0 con 0x7f9000161860 2024-01-19T04:57:52.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.070+0000 7f8ffd7fa640 1 -- 172.21.15.31:0/1328979047 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0032d70 con 0x7f9000161860 2024-01-19T04:57:52.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.070+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90001f6f30 con 0x7f9000161860 2024-01-19T04:57:52.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.071+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90001f6ab0 con 0x7f9000161860 2024-01-19T04:57:52.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.071+0000 7f8ffd7fa640 1 -- 172.21.15.31:0/1328979047 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f8ff003b3f0 con 0x7f9000161860 2024-01-19T04:57:52.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.071+0000 7f8ffd7fa640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8fec06aed0 0x7f8fec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.071+0000 7f8ffd7fa640 1 -- 172.21.15.31:0/1328979047 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ff00ab3e0 con 0x7f9000161860 2024-01-19T04:57:52.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.072+0000 7f90049a4640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8fec06aed0 0x7f8fec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.072+0000 7f90049a4640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8fec06aed0 0x7f8fec06d390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8001460 tx=0x7f8ff8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.076+0000 7f8ffd7fa640 1 -- 172.21.15.31:0/1328979047 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f90001f6ab0 con 0x7f9000161860 2024-01-19T04:57:52.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.253+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f90001f6ab0 con 0x7f8fec06aed0 2024-01-19T04:57:52.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.253+0000 7f8ffd7fa640 1 -- 172.21.15.31:0/1328979047 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f90001f6ab0 con 0x7f8fec06aed0 2024-01-19T04:57:52.253 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8fec06aed0 msgr2=0x7f8fec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8fec06aed0 0x7f8fec06d390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8001460 tx=0x7f8ff8009040 comp rx=0 tx=0).stop 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 msgr2=0x7f9000158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000158be0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00097a0 tx=0x7f8ff00046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 shutdown_connections 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f8fec06aed0 0x7f8fec06d390 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9000161860 0x7f9000158be0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900015d3a0 0x7f90000c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 --2- 172.21.15.31:0/1328979047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f90000cf570 0x7f90001f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 >> 172.21.15.31:0/1328979047 conn(0x7f90000d1100 msgr2=0x7f90000b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 shutdown_connections 2024-01-19T04:57:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.255+0000 7f9006c08640 1 -- 172.21.15.31:0/1328979047 wait complete. 2024-01-19T04:57:52.265 INFO:tasks.ceph:Scrubbing osd.3 2024-01-19T04:57:52.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:52.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.370+0000 7f5f95c82640 1 Processor -- start 2024-01-19T04:57:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.371+0000 7f5f95c82640 1 -- start start 2024-01-19T04:57:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.371+0000 7f5f95c82640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f900d3250 0x7f5f900d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.371+0000 7f5f95c82640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5f900d1720 0x7f5f900d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.371+0000 7f5f95c82640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f900cfd30 con 0x7f5f9015d640 2024-01-19T04:57:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.371+0000 7f5f95c82640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900d09b0 con 0x7f5f900d3250 2024-01-19T04:57:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.371+0000 7f5f95c82640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900b92f0 con 0x7f5f900d1720 2024-01-19T04:57:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.371+0000 7f5f8effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f900d3250 0x7f5f900d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f900d3250 0x7f5f900d36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58136/0 (socket says 172.21.15.31:58136) 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8effd640 1 -- 172.21.15.31:0/2741486069 learned_addr learned my addr 172.21.15.31:0/2741486069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8ffff640 1 --2- 172.21.15.31:0/2741486069 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d1720 0x7f5f900d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8effd640 1 -- 172.21.15.31:0/2741486069 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d1720 msgr2=0x7f5f900d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8effd640 1 --2- 172.21.15.31:0/2741486069 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d1720 0x7f5f900d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8effd640 1 -- 172.21.15.31:0/2741486069 >> v1:172.21.15.137:6789/0 conn(0x7f5f9015d640 legacy=0x7f5f900b8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8effd640 1 -- 172.21.15.31:0/2741486069 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900d0010 con 0x7f5f900d3250 2024-01-19T04:57:52.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.372+0000 7f5f8ffff640 1 --2- 172.21.15.31:0/2741486069 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d1720 0x7f5f900d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f8effd640 1 --2- 172.21.15.31:0/2741486069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f900d36d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5f900d1150 tx=0x7f5f8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97b5c98de0e6e63 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2741486069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84030860 con 0x7f5f900d3250 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2741486069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f840309c0 con 0x7f5f900d3250 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f8e7fc640 1 -- 172.21.15.31:0/2741486069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84030d50 con 0x7f5f900d3250 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 -- 172.21.15.31:0/2741486069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 msgr2=0x7f5f900d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 --2- 172.21.15.31:0/2741486069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f900d36d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5f900d1150 tx=0x7f5f8402b9a0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 -- 172.21.15.31:0/2741486069 shutdown_connections 2024-01-19T04:57:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 --2- 172.21.15.31:0/2741486069 >> v2:172.21.15.137:3300/0 conn(0x7f5f900d1720 0x7f5f900d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 --2- 172.21.15.31:0/2741486069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f900d36d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 -- 172.21.15.31:0/2741486069 >> 172.21.15.31:0/2741486069 conn(0x7f5f90072310 msgr2=0x7f5f9006b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 -- 172.21.15.31:0/2741486069 shutdown_connections 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.373+0000 7f5f95c82640 1 -- 172.21.15.31:0/2741486069 wait complete. 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.374+0000 7f5f95c82640 1 Processor -- start 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f95c82640 1 -- start start 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f95c82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d1720 0x7f5f901575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f95c82640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f90157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f95c82640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f9015d640 0x7f5f901f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f95c82640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90158040 con 0x7f5f900d3250 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f95c82640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f901f3d90 con 0x7f5f900d1720 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f95c82640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900c84b0 con 0x7f5f9015d640 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f8effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f90157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f8effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f90157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58144/0 (socket says 172.21.15.31:58144) 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.375+0000 7f5f8effd640 1 -- 172.21.15.31:0/676115053 learned_addr learned my addr 172.21.15.31:0/676115053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8f7fe640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d1720 0x7f5f901575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8ffff640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f9015d640 0x7f5f901f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8effd640 1 -- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f9015d640 msgr2=0x7f5f901f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8effd640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5f9015d640 0x7f5f901f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8effd640 1 -- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d1720 msgr2=0x7f5f901575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8effd640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f900d1720 0x7f5f901575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8effd640 1 -- 172.21.15.31:0/676115053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f840094d0 con 0x7f5f900d3250 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f8effd640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f90157b00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5f8402b9e0 tx=0x7f5f84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84039d60 con 0x7f5f900d3250 2024-01-19T04:57:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84043400 con 0x7f5f900d3250 2024-01-19T04:57:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f901f4050 con 0x7f5f900d3250 2024-01-19T04:57:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840437b0 con 0x7f5f900d3250 2024-01-19T04:57:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.376+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f901f4610 con 0x7f5f900d3250 2024-01-19T04:57:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.377+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f900d05d0 con 0x7f5f900d3250 2024-01-19T04:57:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f5f84043910 con 0x7f5f900d3250 2024-01-19T04:57:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f77fff640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5f8403e070 con 0x7f5f900d3250 2024-01-19T04:57:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f77fff640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f5f7c070d30 0x7f5f7c073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 --> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f7c073800 con 0x7f5f7c070d30 2024-01-19T04:57:52.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f840ae7e0 con 0x7f5f900d3250 2024-01-19T04:57:52.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f8f7fe640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.378+0000 7f5f8ffff640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f5f7c070d30 0x7f5f7c073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.379+0000 7f5f8ffff640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f5f7c070d30 0x7f5f7c073150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.379+0000 7f5f8f7fe640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5f800016e0 tx=0x7f5f80009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.379+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== osd.3 v2:172.21.15.31:6809/3034750361 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5f7c073800 con 0x7f5f7c070d30 2024-01-19T04:57:52.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.392+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 --> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5f90073d90 con 0x7f5f7c070d30 2024-01-19T04:57:52.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f77fff640 1 -- 172.21.15.31:0/676115053 <== osd.3 v2:172.21.15.31:6809/3034750361 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5f90073d90 con 0x7f5f7c070d30 2024-01-19T04:57:52.394 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:52.395 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:52.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f5f7c070d30 msgr2=0x7f5f7c073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:6809/3034750361,v1:172.21.15.31:6811/3034750361] conn(0x7f5f7c070d30 0x7f5f7c073150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 msgr2=0x7f5f7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f5f7c06aed0 0x7f5f7c06d390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5f800016e0 tx=0x7f5f80009040 comp rx=0 tx=0).stop 2024-01-19T04:57:52.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 msgr2=0x7f5f90157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 --2- 172.21.15.31:0/676115053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f900d3250 0x7f5f90157b00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5f8402b9e0 tx=0x7f5f84004730 comp rx=0 tx=0).stop 2024-01-19T04:57:52.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f8ffff640 1 -- 172.21.15.31:0/676115053 reap_dead start 2024-01-19T04:57:52.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 shutdown_connections 2024-01-19T04:57:52.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 >> 172.21.15.31:0/676115053 conn(0x7f5f90072310 msgr2=0x7f5f900b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 shutdown_connections 2024-01-19T04:57:52.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.393+0000 7f5f95c82640 1 -- 172.21.15.31:0/676115053 wait complete. 2024-01-19T04:57:52.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-19T04:57:52.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.505+0000 7f0ae4c72640 1 Processor -- start 2024-01-19T04:57:52.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.505+0000 7f0ae4c72640 1 -- start start 2024-01-19T04:57:52.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0ae4c72640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae0143440 0x7f0ae00cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0ae4c72640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ae006ddc0 0x7f0ae00b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0ae4c72640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae015ecb0 con 0x7f0ae00b33f0 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0ae4c72640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae0072440 con 0x7f0ae006ddc0 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0ae4c72640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0143820 con 0x7f0ae0143440 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0aded9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ae006ddc0 0x7f0ae00b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0addd9b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0ae00b33f0 0x7f0ae006d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38344/0 (socket says 172.21.15.31:38344) 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0addd9b640 1 -- 172.21.15.31:0/3907036833 learned_addr learned my addr 172.21.15.31:0/3907036833 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0ade59c640 1 --2- 172.21.15.31:0/3907036833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae0143440 0x7f0ae00cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0aded9d640 1 -- 172.21.15.31:0/3907036833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae0143440 msgr2=0x7f0ae00cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:52.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0aded9d640 1 --2- 172.21.15.31:0/3907036833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae0143440 0x7f0ae00cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0aded9d640 1 -- 172.21.15.31:0/3907036833 >> v1:172.21.15.31:6789/0 conn(0x7f0ae00b33f0 legacy=0x7f0ae006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0add59a640 1 -- 172.21.15.31:0/3907036833 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803561724 0 0) 0x7f0ae015ecb0 con 0x7f0ae00b33f0 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.506+0000 7f0aded9d640 1 -- 172.21.15.31:0/3907036833 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae015e130 con 0x7f0ae006ddc0 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.507+0000 7f0aded9d640 1 --2- 172.21.15.31:0/3907036833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae00b21b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009dc0 tx=0x7f0ad002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ccc5cc406b6a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.507+0000 7f0add59a640 1 -- 172.21.15.31:0/3907036833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad002cbb0 con 0x7f0ae006ddc0 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.507+0000 7f0add59a640 1 -- 172.21.15.31:0/3907036833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad002cd10 con 0x7f0ae006ddc0 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.507+0000 7f0add59a640 1 -- 172.21.15.31:0/3907036833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0034660 con 0x7f0ae006ddc0 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3907036833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 msgr2=0x7f0ae00b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3907036833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae00b21b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009dc0 tx=0x7f0ad002be00 comp rx=0 tx=0).stop 2024-01-19T04:57:52.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3907036833 shutdown_connections 2024-01-19T04:57:52.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3907036833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae00b21b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3907036833 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae0143440 0x7f0ae00cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3907036833 >> 172.21.15.31:0/3907036833 conn(0x7f0ae006aa00 msgr2=0x7f0ae006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3907036833 shutdown_connections 2024-01-19T04:57:52.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.508+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3907036833 wait complete. 2024-01-19T04:57:52.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.509+0000 7f0ae4c72640 1 Processor -- start 2024-01-19T04:57:52.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.509+0000 7f0ae4c72640 1 -- start start 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ae4c72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ae4c72640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae00b33f0 0x7f0ae00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ae4c72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0143440 0x7f0ae00ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ae4c72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00ad710 con 0x7f0ae006ddc0 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ae4c72640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae013f300 con 0x7f0ae0143440 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ae4c72640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00ad430 con 0x7f0ae00b33f0 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae015aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58160/0 (socket says 172.21.15.31:58160) 2024-01-19T04:57:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 -- 172.21.15.31:0/3731446056 learned_addr learned my addr 172.21.15.31:0/3731446056 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0addd9b640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae00b33f0 0x7f0ae00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0aded9d640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0143440 0x7f0ae00ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 -- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae00b33f0 msgr2=0x7f0ae00ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae00b33f0 0x7f0ae00ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 -- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0143440 msgr2=0x7f0ae00ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0143440 0x7f0ae00ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 -- 172.21.15.31:0/3731446056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0009a70 con 0x7f0ae006ddc0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.510+0000 7f0ade59c640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae015aa00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800d800 tx=0x7f0ad800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.511+0000 7f0aceffd640 1 -- 172.21.15.31:0/3731446056 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800b6c0 con 0x7f0ae006ddc0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.511+0000 7f0aceffd640 1 -- 172.21.15.31:0/3731446056 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad800b820 con 0x7f0ae006ddc0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.511+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae01f31c0 con 0x7f0ae006ddc0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.511+0000 7f0aceffd640 1 -- 172.21.15.31:0/3731446056 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800bbb0 con 0x7f0ae006ddc0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.511+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01f3780 con 0x7f0ae006ddc0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.512+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae0074900 con 0x7f0ae006ddc0 2024-01-19T04:57:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.512+0000 7f0aceffd640 1 -- 172.21.15.31:0/3731446056 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f0ad800bd10 con 0x7f0ae006ddc0 2024-01-19T04:57:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.512+0000 7f0aceffd640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0ac006aed0 0x7f0ac006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.512+0000 7f0aceffd640 1 -- 172.21.15.31:0/3731446056 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0ad80898d0 con 0x7f0ae006ddc0 2024-01-19T04:57:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.513+0000 7f0addd9b640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0ac006aed0 0x7f0ac006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.513+0000 7f0addd9b640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0ac006aed0 0x7f0ac006d390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00adcb0 tx=0x7f0ad4007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.517+0000 7f0aceffd640 1 -- 172.21.15.31:0/3731446056 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0ad8054aa0 con 0x7f0ae006ddc0 2024-01-19T04:57:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.694+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ae0066f60 con 0x7f0ac006aed0 2024-01-19T04:57:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.695+0000 7f0aceffd640 1 -- 172.21.15.31:0/3731446056 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0ae0066f60 con 0x7f0ac006aed0 2024-01-19T04:57:52.695 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2024-01-19T04:57:52.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0ac006aed0 msgr2=0x7f0ac006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0ac006aed0 0x7f0ac006d390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00adcb0 tx=0x7f0ad4007b50 comp rx=0 tx=0).stop 2024-01-19T04:57:52.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 msgr2=0x7f0ae015aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae015aa00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800d800 tx=0x7f0ad800dcc0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 shutdown_connections 2024-01-19T04:57:52.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f0ac006aed0 0x7f0ac006d390 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0143440 0x7f0ae00ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0ae00b33f0 0x7f0ae00ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 --2- 172.21.15.31:0/3731446056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae006ddc0 0x7f0ae015aa00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 >> 172.21.15.31:0/3731446056 conn(0x7f0ae006aa00 msgr2=0x7f0ae013f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 shutdown_connections 2024-01-19T04:57:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.697+0000 7f0ae4c72640 1 -- 172.21.15.31:0/3731446056 wait complete. 2024-01-19T04:57:52.707 INFO:tasks.ceph:Scrubbing osd.4 2024-01-19T04:57:52.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.812+0000 7f6be79bb640 1 Processor -- start 2024-01-19T04:57:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.812+0000 7f6be79bb640 1 -- start start 2024-01-19T04:57:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be79bb640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6be01619f0 0x7f6be006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be79bb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6be0161000 0x7f6be006d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be79bb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be0157f40 con 0x7f6be00cd900 2024-01-19T04:57:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be79bb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00b2c40 con 0x7f6be0161000 2024-01-19T04:57:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be79bb640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00b2f20 con 0x7f6be01619f0 2024-01-19T04:57:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be4f56640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6be0161000 0x7f6be006d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be5f58640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6be00cd900 0x7f6be00cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38354/0 (socket says 172.21.15.31:38354) 2024-01-19T04:57:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.813+0000 7f6be5f58640 1 -- 172.21.15.31:0/82515774 learned_addr learned my addr 172.21.15.31:0/82515774 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6be5757640 1 --2- 172.21.15.31:0/82515774 >> v2:172.21.15.137:3300/0 conn(0x7f6be01619f0 0x7f6be006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6be4f56640 1 -- 172.21.15.31:0/82515774 >> v2:172.21.15.137:3300/0 conn(0x7f6be01619f0 msgr2=0x7f6be006cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6bdffff640 1 -- 172.21.15.31:0/82515774 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710763503 0 0) 0x7f6be0157f40 con 0x7f6be00cd900 2024-01-19T04:57:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6be4f56640 1 --2- 172.21.15.31:0/82515774 >> v2:172.21.15.137:3300/0 conn(0x7f6be01619f0 0x7f6be006cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6be4f56640 1 -- 172.21.15.31:0/82515774 >> v1:172.21.15.31:6789/0 conn(0x7f6be00cd900 legacy=0x7f6be00cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6be4f56640 1 -- 172.21.15.31:0/82515774 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be006f870 con 0x7f6be0161000 2024-01-19T04:57:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6be5757640 1 --2- 172.21.15.31:0/82515774 >> v2:172.21.15.137:3300/0 conn(0x7f6be01619f0 0x7f6be006cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.814+0000 7f6be4f56640 1 --2- 172.21.15.31:0/82515774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be0161000 0x7f6be006d2f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6be00d1080 tx=0x7f6bd402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1084267a69abef server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6bdffff640 1 -- 172.21.15.31:0/82515774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4031830 con 0x7f6be0161000 2024-01-19T04:57:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6bdffff640 1 -- 172.21.15.31:0/82515774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4031990 con 0x7f6be0161000 2024-01-19T04:57:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6bdffff640 1 -- 172.21.15.31:0/82515774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4031d20 con 0x7f6be0161000 2024-01-19T04:57:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 -- 172.21.15.31:0/82515774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be0161000 msgr2=0x7f6be006d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 --2- 172.21.15.31:0/82515774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be0161000 0x7f6be006d2f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6be00d1080 tx=0x7f6bd402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 -- 172.21.15.31:0/82515774 shutdown_connections 2024-01-19T04:57:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 --2- 172.21.15.31:0/82515774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be0161000 0x7f6be006d2f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 --2- 172.21.15.31:0/82515774 >> v2:172.21.15.137:3300/0 conn(0x7f6be01619f0 0x7f6be006cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 -- 172.21.15.31:0/82515774 >> 172.21.15.31:0/82515774 conn(0x7f6be0066340 msgr2=0x7f6be0066720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 -- 172.21.15.31:0/82515774 shutdown_connections 2024-01-19T04:57:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.815+0000 7f6be79bb640 1 -- 172.21.15.31:0/82515774 wait complete. 2024-01-19T04:57:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.816+0000 7f6be79bb640 1 Processor -- start 2024-01-19T04:57:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.816+0000 7f6be79bb640 1 -- start start 2024-01-19T04:57:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be79bb640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6be00cd900 0x7f6be01515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be79bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be0161000 0x7f6be00b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be79bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be01619f0 0x7f6be0159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be79bb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b5c40 con 0x7f6be01619f0 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be79bb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b52a0 con 0x7f6be0161000 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be79bb640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b5580 con 0x7f6be00cd900 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be5f58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be01619f0 0x7f6be0159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be5f58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be01619f0 0x7f6be0159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58174/0 (socket says 172.21.15.31:58174) 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.817+0000 7f6be5f58640 1 -- 172.21.15.31:0/952171336 learned_addr learned my addr 172.21.15.31:0/952171336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5f58640 1 -- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6be00cd900 msgr2=0x7f6be01515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5757640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6be00cd900 0x7f6be01515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be4f56640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be0161000 0x7f6be00b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5f58640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6be00cd900 0x7f6be01515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5f58640 1 -- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be0161000 msgr2=0x7f6be00b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5f58640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be0161000 0x7f6be00b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5f58640 1 -- 172.21.15.31:0/952171336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd4009400 con 0x7f6be01619f0 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5757640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6be00cd900 0x7f6be01515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be4f56640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be0161000 0x7f6be00b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be5f58640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be01619f0 0x7f6be0159670 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800c410 tx=0x7f6bd800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80074e0 con 0x7f6be01619f0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd8007640 con 0x7f6be01619f0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8004a20 con 0x7f6be01619f0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be00b5860 con 0x7f6be01619f0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.818+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be0157450 con 0x7f6be01619f0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.819+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6be00b5410 con 0x7f6be01619f0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.819+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f6bd8004b80 con 0x7f6be01619f0 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.820+0000 7f6bddffb640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6bcc06af00 0x7f6bcc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.820+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6bd8089070 con 0x7f6be01619f0 2024-01-19T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.820+0000 7f6bddffb640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f6bcc070d60 0x7f6bcc073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.820+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 --> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bcc073830 con 0x7f6bcc070d60 2024-01-19T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.820+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bd808b730 con 0x7f6be01619f0 2024-01-19T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.820+0000 7f6be5757640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6bcc06af00 0x7f6bcc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.820+0000 7f6be4f56640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f6bcc070d60 0x7f6bcc073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.821+0000 7f6be5757640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6bcc06af00 0x7f6bcc06d3c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6be01569f0 tx=0x7f6bd0007550 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.821+0000 7f6be4f56640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f6bcc070d60 0x7f6bcc073180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.821+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== osd.4 v2:172.21.15.137:6814/7950289 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6bcc073830 con 0x7f6bcc070d60 2024-01-19T04:57:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.835+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 --> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6be00b37c0 con 0x7f6bcc070d60 2024-01-19T04:57:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.835+0000 7f6bddffb640 1 -- 172.21.15.31:0/952171336 <== osd.4 v2:172.21.15.137:6814/7950289 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6bd4004040 con 0x7f6bcc070d60 2024-01-19T04:57:52.835 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:52.835 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:52.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f6bcc070d60 msgr2=0x7f6bcc073180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6814/7950289,v1:172.21.15.137:6817/7950289] conn(0x7f6bcc070d60 0x7f6bcc073180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6bcc06af00 msgr2=0x7f6bcc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f6bcc06af00 0x7f6bcc06d3c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6be01569f0 tx=0x7f6bd0007550 comp rx=0 tx=0).stop 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be01619f0 msgr2=0x7f6be0159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 --2- 172.21.15.31:0/952171336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be01619f0 0x7f6be0159670 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800c410 tx=0x7f6bd800c8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be5f58640 1 -- 172.21.15.31:0/952171336 reap_dead start 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 shutdown_connections 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 >> 172.21.15.31:0/952171336 conn(0x7f6be0066340 msgr2=0x7f6be0066720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 shutdown_connections 2024-01-19T04:57:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.836+0000 7f6be79bb640 1 -- 172.21.15.31:0/952171336 wait complete. 2024-01-19T04:57:52.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-19T04:57:52.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.948+0000 7fc759525640 1 Processor -- start 2024-01-19T04:57:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.948+0000 7fc759525640 1 -- start start 2024-01-19T04:57:52.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.948+0000 7fc759525640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc7540cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.948+0000 7fc759525640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc7540b33f0 0x7fc7540b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.948+0000 7fc759525640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75415ecb0 con 0x7fc754143440 2024-01-19T04:57:52.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.948+0000 7fc759525640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc754072440 con 0x7fc7540b33f0 2024-01-19T04:57:52.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.948+0000 7fc759525640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc754143820 con 0x7fc75406ddc0 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc7540cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc7537fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc7540b33f0 0x7fc7540b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc7540cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.31:42628/0 (socket says 172.21.15.31:42628) 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc74bfff640 1 -- 172.21.15.31:0/267038937 learned_addr learned my addr 172.21.15.31:0/267038937 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc74bfff640 1 -- 172.21.15.31:0/267038937 >> v2:172.21.15.137:3300/0 conn(0x7fc7540b33f0 msgr2=0x7fc7540b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc74bfff640 1 --2- 172.21.15.31:0/267038937 >> v2:172.21.15.137:3300/0 conn(0x7fc7540b33f0 0x7fc7540b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc74bfff640 1 -- 172.21.15.31:0/267038937 >> v1:172.21.15.137:6789/0 conn(0x7fc754143440 legacy=0x7fc75406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc74bfff640 1 -- 172.21.15.31:0/267038937 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75415e130 con 0x7fc75406ddc0 2024-01-19T04:57:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.949+0000 7fc7537fe640 1 --2- 172.21.15.31:0/267038937 >> v2:172.21.15.137:3300/0 conn(0x7fc7540b33f0 0x7fc7540b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.950+0000 7fc74bfff640 1 --2- 172.21.15.31:0/267038937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc7540cc720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009870 tx=0x7fc73c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86ff14478d48544 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.950+0000 7fc7527fc640 1 -- 172.21.15.31:0/267038937 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c031640 con 0x7fc75406ddc0 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.950+0000 7fc7527fc640 1 -- 172.21.15.31:0/267038937 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c0317a0 con 0x7fc75406ddc0 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.950+0000 7fc7527fc640 1 -- 172.21.15.31:0/267038937 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c031b30 con 0x7fc75406ddc0 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 -- 172.21.15.31:0/267038937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 msgr2=0x7fc7540cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 --2- 172.21.15.31:0/267038937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc7540cc720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009870 tx=0x7fc73c02b8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 -- 172.21.15.31:0/267038937 shutdown_connections 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 --2- 172.21.15.31:0/267038937 >> v2:172.21.15.137:3300/0 conn(0x7fc7540b33f0 0x7fc7540b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 --2- 172.21.15.31:0/267038937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc7540cc720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 -- 172.21.15.31:0/267038937 >> 172.21.15.31:0/267038937 conn(0x7fc75406aa00 msgr2=0x7fc75406ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 -- 172.21.15.31:0/267038937 shutdown_connections 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.951+0000 7fc759525640 1 -- 172.21.15.31:0/267038937 wait complete. 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.952+0000 7fc759525640 1 Processor -- start 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.952+0000 7fc759525640 1 -- start start 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc759525640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc754142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc759525640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7540b33f0 0x7fc75413c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc759525640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc754143440 0x7fc754140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc759525640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc754141290 con 0x7fc7540b33f0 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc759525640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc754141570 con 0x7fc754143440 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc759525640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc754141850 con 0x7fc75406ddc0 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7540b33f0 0x7fc75413c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7540b33f0 0x7fc75413c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58190/0 (socket says 172.21.15.31:58190) 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 -- 172.21.15.31:0/590340815 learned_addr learned my addr 172.21.15.31:0/590340815 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc752ffd640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc754142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc7537fe640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc754143440 0x7fc754140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 -- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 msgr2=0x7fc754142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc754142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 -- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc754143440 msgr2=0x7fc754140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc754143440 0x7fc754140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 -- 172.21.15.31:0/590340815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c009a70 con 0x7fc7540b33f0 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc7537fe640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc754143440 0x7fc754140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc752ffd640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc754142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.953+0000 7fc74bfff640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7540b33f0 0x7fc75413c6a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc73c02b930 tx=0x7fc73c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.954+0000 7fc74b7fe640 1 -- 172.21.15.31:0/590340815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c03a930 con 0x7fc7540b33f0 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.954+0000 7fc74b7fe640 1 -- 172.21.15.31:0/590340815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c03aab0 con 0x7fc7540b33f0 2024-01-19T04:57:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.954+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73c0094d0 con 0x7fc7540b33f0 2024-01-19T04:57:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.954+0000 7fc74b7fe640 1 -- 172.21.15.31:0/590340815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c03ae60 con 0x7fc7540b33f0 2024-01-19T04:57:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.954+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc754141e10 con 0x7fc7540b33f0 2024-01-19T04:57:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.955+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc754074900 con 0x7fc7540b33f0 2024-01-19T04:57:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.955+0000 7fc74b7fe640 1 -- 172.21.15.31:0/590340815 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7fc73c02f050 con 0x7fc7540b33f0 2024-01-19T04:57:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.955+0000 7fc74b7fe640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc73406aed0 0x7fc73406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.955+0000 7fc74b7fe640 1 -- 172.21.15.31:0/590340815 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc73c0ab0f0 con 0x7fc7540b33f0 2024-01-19T04:57:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.956+0000 7fc752ffd640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc73406aed0 0x7fc73406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:52.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.956+0000 7fc752ffd640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc73406aed0 0x7fc73406d390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc744001660 tx=0x7fc744009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:52.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:52.960+0000 7fc74b7fe640 1 -- 172.21.15.31:0/590340815 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc73c076350 con 0x7fc7540b33f0 2024-01-19T04:57:53.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.136+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc75415e9d0 con 0x7fc73406aed0 2024-01-19T04:57:53.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.136+0000 7fc74b7fe640 1 -- 172.21.15.31:0/590340815 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc744001350 con 0x7fc73406aed0 2024-01-19T04:57:53.136 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2024-01-19T04:57:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc73406aed0 msgr2=0x7fc73406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc73406aed0 0x7fc73406d390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc744001660 tx=0x7fc744009040 comp rx=0 tx=0).stop 2024-01-19T04:57:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7540b33f0 msgr2=0x7fc75413c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7540b33f0 0x7fc75413c6a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc73c02b930 tx=0x7fc73c0046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 shutdown_connections 2024-01-19T04:57:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7fc73406aed0 0x7fc73406d390 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc754143440 0x7fc754140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7540b33f0 0x7fc75413c6a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 --2- 172.21.15.31:0/590340815 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc75406ddc0 0x7fc754142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 >> 172.21.15.31:0/590340815 conn(0x7fc75406aa00 msgr2=0x7fc7540cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 shutdown_connections 2024-01-19T04:57:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.138+0000 7fc759525640 1 -- 172.21.15.31:0/590340815 wait complete. 2024-01-19T04:57:53.148 INFO:tasks.ceph:Scrubbing osd.5 2024-01-19T04:57:53.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.254+0000 7f5067f48640 1 Processor -- start 2024-01-19T04:57:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.254+0000 7f5067f48640 1 -- start start 2024-01-19T04:57:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f5067f48640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f506015d3b0 0x7f50600d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f5067f48640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50600b5450 0x7f50600b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f5067f48640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600d1d20 con 0x7f50600b9010 2024-01-19T04:57:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f5067f48640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50600d2000 con 0x7f50600b5450 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f5067f48640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50600d22e0 con 0x7f506015d3b0 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50664e5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50600b5450 0x7f50600b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50664e5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f50600b5450 0x7f50600b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58200/0 (socket says 172.21.15.31:58200) 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50664e5640 1 -- 172.21.15.31:0/3758197746 learned_addr learned my addr 172.21.15.31:0/3758197746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50654e3640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f50600b9010 0x7f50600b4d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38356/0 (socket says 172.21.15.31:38356) 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f5065ce4640 1 --2- 172.21.15.31:0/3758197746 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f506015d3b0 0x7f50600d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50664e5640 1 -- 172.21.15.31:0/3758197746 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f506015d3b0 msgr2=0x7f50600d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50664e5640 1 --2- 172.21.15.31:0/3758197746 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f506015d3b0 0x7f50600d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50664e5640 1 -- 172.21.15.31:0/3758197746 >> v1:172.21.15.31:6789/0 conn(0x7f50600b9010 legacy=0x7f50600b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.255+0000 7f50664e5640 1 -- 172.21.15.31:0/3758197746 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600d25c0 con 0x7f50600b5450 2024-01-19T04:57:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.256+0000 7f50664e5640 1 --2- 172.21.15.31:0/3758197746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50600b63f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5058009e50 tx=0x7f505802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c30fe3a9843c7c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.256+0000 7f5064ce2640 1 -- 172.21.15.31:0/3758197746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505802cb00 con 0x7f50600b5450 2024-01-19T04:57:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.256+0000 7f5064ce2640 1 -- 172.21.15.31:0/3758197746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505802cc60 con 0x7f50600b5450 2024-01-19T04:57:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.256+0000 7f5064ce2640 1 -- 172.21.15.31:0/3758197746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580343f0 con 0x7f50600b5450 2024-01-19T04:57:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.256+0000 7f5067f48640 1 -- 172.21.15.31:0/3758197746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 msgr2=0x7f50600b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.256+0000 7f5067f48640 1 --2- 172.21.15.31:0/3758197746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50600b63f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5058009e50 tx=0x7f505802bf60 comp rx=0 tx=0).stop 2024-01-19T04:57:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.257+0000 7f5067f48640 1 -- 172.21.15.31:0/3758197746 shutdown_connections 2024-01-19T04:57:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.257+0000 7f5067f48640 1 --2- 172.21.15.31:0/3758197746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50600b63f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.257+0000 7f5067f48640 1 --2- 172.21.15.31:0/3758197746 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f506015d3b0 0x7f50600d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.257+0000 7f5067f48640 1 -- 172.21.15.31:0/3758197746 >> 172.21.15.31:0/3758197746 conn(0x7f50600cfd40 msgr2=0x7f50600d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.257+0000 7f5067f48640 1 -- 172.21.15.31:0/3758197746 shutdown_connections 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.257+0000 7f5067f48640 1 -- 172.21.15.31:0/3758197746 wait complete. 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 Processor -- start 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 -- start start 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50601608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50600b9010 0x7f50600aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506015d3b0 0x7f506015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5060159340 con 0x7f50600b5450 2024-01-19T04:57:53.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506015f840 con 0x7f506015d3b0 2024-01-19T04:57:53.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.258+0000 7f5067f48640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506015fb20 con 0x7f50600b9010 2024-01-19T04:57:53.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50601608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50601608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58206/0 (socket says 172.21.15.31:58206) 2024-01-19T04:57:53.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 -- 172.21.15.31:0/3541828662 learned_addr learned my addr 172.21.15.31:0/3541828662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f50654e3640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50600b9010 0x7f50600aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f50664e5640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506015d3b0 0x7f506015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 -- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50600b9010 msgr2=0x7f50600aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f50600b9010 0x7f50600aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 -- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506015d3b0 msgr2=0x7f506015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506015d3b0 0x7f506015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 -- 172.21.15.31:0/3541828662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058009b30 con 0x7f50600b5450 2024-01-19T04:57:53.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f50664e5640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506015d3b0 0x7f506015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5065ce4640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50601608e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c7f0 tx=0x7f505c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c004220 con 0x7f50600b5450 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c004380 con 0x7f50600b5450 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506015fe00 con 0x7f50600b5450 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c007840 con 0x7f50600b5450 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.259+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50601603c0 con 0x7f50600b5450 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.260+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f506015cbc0 con 0x7f50600b5450 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f505c007bb0 con 0x7f50600b5450 2024-01-19T04:57:53.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50567fc640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f504406aed0 0x7f504406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f505c089100 con 0x7f50600b5450 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50567fc640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f5044070d30 0x7f5044073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 --> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5044073800 con 0x7f5044070d30 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f505c08b7c0 con 0x7f50600b5450 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50654e3640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f504406aed0 0x7f504406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.261+0000 7f50664e5640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f5044070d30 0x7f5044073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.262+0000 7f50654e3640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f504406aed0 0x7f504406d390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f50601f6890 tx=0x7f5050007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.262+0000 7f50664e5640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f5044070d30 0x7f5044073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.263+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== osd.5 v2:172.21.15.137:6809/4102773142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5044073800 con 0x7f5044070d30 2024-01-19T04:57:53.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 --> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f50600d18f0 con 0x7f5044070d30 2024-01-19T04:57:53.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f50567fc640 1 -- 172.21.15.31:0/3541828662 <== osd.5 v2:172.21.15.137:6809/4102773142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5058040410 con 0x7f5044070d30 2024-01-19T04:57:53.276 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:53.276 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:53.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f5044070d30 msgr2=0x7f5044073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f5067f48640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6809/4102773142,v1:172.21.15.137:6811/4102773142] conn(0x7f5044070d30 0x7f5044073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f504406aed0 msgr2=0x7f504406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f5067f48640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f504406aed0 0x7f504406d390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f50601f6890 tx=0x7f5050007e80 comp rx=0 tx=0).stop 2024-01-19T04:57:53.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 msgr2=0x7f50601608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f5067f48640 1 --2- 172.21.15.31:0/3541828662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50600b5450 0x7f50601608e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c7f0 tx=0x7f505c00ccb0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.276+0000 7f50664e5640 1 -- 172.21.15.31:0/3541828662 reap_dead start 2024-01-19T04:57:53.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.277+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 shutdown_connections 2024-01-19T04:57:53.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.277+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 >> 172.21.15.31:0/3541828662 conn(0x7f50600cfd40 msgr2=0x7f50600b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.277+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 shutdown_connections 2024-01-19T04:57:53.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.277+0000 7f5067f48640 1 -- 172.21.15.31:0/3541828662 wait complete. 2024-01-19T04:57:53.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-19T04:57:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.388+0000 7ff531c24640 1 Processor -- start 2024-01-19T04:57:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.388+0000 7ff531c24640 1 -- start start 2024-01-19T04:57:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff531c24640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff52c0d18e0 0x7ff52c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff531c24640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff52c0d3e50 0x7ff52c0d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff531c24640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c0d2200 con 0x7ff52c0d0ef0 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff531c24640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c15dee0 con 0x7ff52c0d3e50 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff531c24640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c15e1c0 con 0x7ff52c0d18e0 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff52c0d3e50 0x7ff52c0d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff52c0d0ef0 0x7ff52c161100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38362/0 (socket says 172.21.15.31:38362) 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52bfff640 1 -- 172.21.15.31:0/1826193770 learned_addr learned my addr 172.21.15.31:0/1826193770 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52b7fe640 1 --2- 172.21.15.31:0/1826193770 >> v2:172.21.15.137:3300/0 conn(0x7ff52c0d18e0 0x7ff52c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52affd640 1 -- 172.21.15.31:0/1826193770 >> v2:172.21.15.137:3300/0 conn(0x7ff52c0d18e0 msgr2=0x7ff52c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52affd640 1 --2- 172.21.15.31:0/1826193770 >> v2:172.21.15.137:3300/0 conn(0x7ff52c0d18e0 0x7ff52c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52affd640 1 -- 172.21.15.31:0/1826193770 >> v1:172.21.15.31:6789/0 conn(0x7ff52c0d0ef0 legacy=0x7ff52c161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52affd640 1 -- 172.21.15.31:0/1826193770 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c15e4a0 con 0x7ff52c0d3e50 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.389+0000 7ff52a7fc640 1 -- 172.21.15.31:0/1826193770 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895684921 0 0) 0x7ff52c0d2200 con 0x7ff52c0d0ef0 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.390+0000 7ff52affd640 1 --2- 172.21.15.31:0/1826193770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d3e50 0x7ff52c0d1cc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0d3280 tx=0x7ff52002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9063d818d88dd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.390+0000 7ff52a7fc640 1 -- 172.21.15.31:0/1826193770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520031830 con 0x7ff52c0d3e50 2024-01-19T04:57:53.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.390+0000 7ff52a7fc640 1 -- 172.21.15.31:0/1826193770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520031990 con 0x7ff52c0d3e50 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.390+0000 7ff52a7fc640 1 -- 172.21.15.31:0/1826193770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520031d20 con 0x7ff52c0d3e50 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 -- 172.21.15.31:0/1826193770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d3e50 msgr2=0x7ff52c0d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 --2- 172.21.15.31:0/1826193770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d3e50 0x7ff52c0d1cc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0d3280 tx=0x7ff52002b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 -- 172.21.15.31:0/1826193770 shutdown_connections 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 --2- 172.21.15.31:0/1826193770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d3e50 0x7ff52c0d1cc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 --2- 172.21.15.31:0/1826193770 >> v2:172.21.15.137:3300/0 conn(0x7ff52c0d18e0 0x7ff52c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 -- 172.21.15.31:0/1826193770 >> 172.21.15.31:0/1826193770 conn(0x7ff52c09c600 msgr2=0x7ff52c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 -- 172.21.15.31:0/1826193770 shutdown_connections 2024-01-19T04:57:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.391+0000 7ff531c24640 1 -- 172.21.15.31:0/1826193770 wait complete. 2024-01-19T04:57:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.392+0000 7ff531c24640 1 Processor -- start 2024-01-19T04:57:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.392+0000 7ff531c24640 1 -- start start 2024-01-19T04:57:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff531c24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff52c0d0ef0 0x7ff52c15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff531c24640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d18e0 0x7ff52c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff531c24640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff52c0d3e50 0x7ff52c14c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff531c24640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c9be0 con 0x7ff52c0d18e0 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff531c24640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c8fd0 con 0x7ff52c0d0ef0 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff531c24640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c92b0 con 0x7ff52c0d3e50 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d18e0 0x7ff52c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d18e0 0x7ff52c15ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58230/0 (socket says 172.21.15.31:58230) 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 -- 172.21.15.31:0/3612443640 learned_addr learned my addr 172.21.15.31:0/3612443640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52b7fe640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff52c0d0ef0 0x7ff52c15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52bfff640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff52c0d3e50 0x7ff52c14c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 -- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff52c0d3e50 msgr2=0x7ff52c14c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff52c0d3e50 0x7ff52c14c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 -- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff52c0d0ef0 msgr2=0x7ff52c15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff52c0d0ef0 0x7ff52c15b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 -- 172.21.15.31:0/3612443640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520009400 con 0x7ff52c0d18e0 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.393+0000 7ff52affd640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d18e0 0x7ff52c15ba20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff520009890 tx=0x7ff5200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.394+0000 7ff513fff640 1 -- 172.21.15.31:0/3612443640 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52003ba70 con 0x7ff52c0d18e0 2024-01-19T04:57:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.394+0000 7ff513fff640 1 -- 172.21.15.31:0/3612443640 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52003bbf0 con 0x7ff52c0d18e0 2024-01-19T04:57:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.394+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c0c9590 con 0x7ff52c0d18e0 2024-01-19T04:57:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.394+0000 7ff513fff640 1 -- 172.21.15.31:0/3612443640 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520045400 con 0x7ff52c0d18e0 2024-01-19T04:57:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.394+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c150100 con 0x7ff52c0d18e0 2024-01-19T04:57:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.395+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c074900 con 0x7ff52c0d18e0 2024-01-19T04:57:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.395+0000 7ff513fff640 1 -- 172.21.15.31:0/3612443640 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff52004e440 con 0x7ff52c0d18e0 2024-01-19T04:57:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.395+0000 7ff513fff640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff51806aed0 0x7ff51806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.395+0000 7ff513fff640 1 -- 172.21.15.31:0/3612443640 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff5200ab590 con 0x7ff52c0d18e0 2024-01-19T04:57:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.396+0000 7ff52b7fe640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff51806aed0 0x7ff51806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.396+0000 7ff52b7fe640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff51806aed0 0x7ff51806d390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006fd0 tx=0x7ff51c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.400+0000 7ff513fff640 1 -- 172.21.15.31:0/3612443640 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff520076760 con 0x7ff52c0d18e0 2024-01-19T04:57:53.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.578+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff52c066710 con 0x7ff51806aed0 2024-01-19T04:57:53.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.579+0000 7ff513fff640 1 -- 172.21.15.31:0/3612443640 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff52c066710 con 0x7ff51806aed0 2024-01-19T04:57:53.578 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2024-01-19T04:57:53.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff51806aed0 msgr2=0x7ff51806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff51806aed0 0x7ff51806d390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006fd0 tx=0x7ff51c008040 comp rx=0 tx=0).stop 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d18e0 msgr2=0x7ff52c15ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d18e0 0x7ff52c15ba20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff520009890 tx=0x7ff5200046c0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 shutdown_connections 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ff51806aed0 0x7ff51806d390 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff52c0d3e50 0x7ff52c14c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff52c0d18e0 0x7ff52c15ba20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 --2- 172.21.15.31:0/3612443640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff52c0d0ef0 0x7ff52c15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 >> 172.21.15.31:0/3612443640 conn(0x7ff52c09c600 msgr2=0x7ff52c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 shutdown_connections 2024-01-19T04:57:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.581+0000 7ff531c24640 1 -- 172.21.15.31:0/3612443640 wait complete. 2024-01-19T04:57:53.591 INFO:tasks.ceph:Scrubbing osd.6 2024-01-19T04:57:53.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:53.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.696+0000 7f8456696640 1 Processor -- start 2024-01-19T04:57:53.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.696+0000 7f8456696640 1 -- start start 2024-01-19T04:57:53.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f8456696640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84500cc100 0x7f845006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f8456696640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845015f340 con 0x7f84500cc4e0 2024-01-19T04:57:53.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f8456696640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845015f620 con 0x7f845015eda0 2024-01-19T04:57:53.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f8456696640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845015f900 con 0x7f84500cc100 2024-01-19T04:57:53.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f844ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84500cc100 0x7f845006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f8454c33640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f84500cc4e0 0x7f84500ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38378/0 (socket says 172.21.15.31:38378) 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f844ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84500cc100 0x7f845006a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58236/0 (socket says 172.21.15.31:58236) 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f8454c33640 1 -- 172.21.15.31:0/3299546313 learned_addr learned my addr 172.21.15.31:0/3299546313 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f844ffff640 1 -- 172.21.15.31:0/3299546313 >> v1:172.21.15.137:6789/0 conn(0x7f845015eda0 legacy=0x7f845015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f844ffff640 1 -- 172.21.15.31:0/3299546313 >> v1:172.21.15.31:6789/0 conn(0x7f84500cc4e0 legacy=0x7f84500ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f844ffff640 1 -- 172.21.15.31:0/3299546313 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845015fbe0 con 0x7f84500cc100 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.697+0000 7f844effd640 1 -- 172.21.15.31:0/3299546313 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629172008 0 0) 0x7f845015f340 con 0x7f84500cc4e0 2024-01-19T04:57:53.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.698+0000 7f844ffff640 1 --2- 172.21.15.31:0/3299546313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc100 0x7f845006a5b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f84440097a0 tx=0x7f844402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edc0ab6851825cd server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.698+0000 7f844effd640 1 -- 172.21.15.31:0/3299546313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844402e070 con 0x7f84500cc100 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.698+0000 7f844effd640 1 -- 172.21.15.31:0/3299546313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84440315a0 con 0x7f84500cc100 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.698+0000 7f844effd640 1 -- 172.21.15.31:0/3299546313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444031930 con 0x7f84500cc100 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.699+0000 7f8456696640 1 -- 172.21.15.31:0/3299546313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc100 msgr2=0x7f845006a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.699+0000 7f8456696640 1 --2- 172.21.15.31:0/3299546313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc100 0x7f845006a5b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f84440097a0 tx=0x7f844402b840 comp rx=0 tx=0).stop 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.699+0000 7f8456696640 1 -- 172.21.15.31:0/3299546313 shutdown_connections 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.699+0000 7f8456696640 1 --2- 172.21.15.31:0/3299546313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc100 0x7f845006a5b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.699+0000 7f8456696640 1 -- 172.21.15.31:0/3299546313 >> 172.21.15.31:0/3299546313 conn(0x7f84500b1990 msgr2=0x7f84500cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.699+0000 7f8456696640 1 -- 172.21.15.31:0/3299546313 shutdown_connections 2024-01-19T04:57:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.699+0000 7f8456696640 1 -- 172.21.15.31:0/3299546313 wait complete. 2024-01-19T04:57:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.700+0000 7f8456696640 1 Processor -- start 2024-01-19T04:57:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.700+0000 7f8456696640 1 -- start start 2024-01-19T04:57:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8456696640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84500cc100 0x7f845013d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8456696640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc4e0 0x7f84501f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8456696640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f845015eda0 0x7f84501f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8456696640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845013e320 con 0x7f84500cc4e0 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8456696640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845015b6b0 con 0x7f84500cc100 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8456696640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845013dd60 con 0x7f845015eda0 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc4e0 0x7f84501f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc4e0 0x7f84501f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58252/0 (socket says 172.21.15.31:58252) 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 -- 172.21.15.31:0/2730178583 learned_addr learned my addr 172.21.15.31:0/2730178583 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844ffff640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84500cc100 0x7f845013d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 -- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f845015eda0 msgr2=0x7f84501f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8454c33640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f845015eda0 0x7f84501f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f845015eda0 0x7f84501f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 -- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84500cc100 msgr2=0x7f845013d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84500cc100 0x7f845013d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 -- 172.21.15.31:0/2730178583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8444009400 con 0x7f84500cc4e0 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844ffff640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84500cc100 0x7f845013d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f8454c33640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f845015eda0 0x7f84501f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.701+0000 7f844f7fe640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc4e0 0x7f84501f4390 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f844000c430 tx=0x7f844000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.702+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440011790 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.702+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84400118f0 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.702+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440011ce0 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.702+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845013e040 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.702+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8450156220 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.702+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8450155ab0 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.703+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f844001c460 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.703+0000 7f844cff9640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f843c06af00 0x7f843c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.703+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8440089270 con 0x7f84500cc4e0 2024-01-19T04:57:53.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.704+0000 7f844cff9640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f843c070d60 0x7f843c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.704+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 --> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f843c073830 con 0x7f843c070d60 2024-01-19T04:57:53.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.704+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f844008b850 con 0x7f84500cc4e0 2024-01-19T04:57:53.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.704+0000 7f844ffff640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f843c06af00 0x7f843c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.704+0000 7f8454c33640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f843c070d60 0x7f843c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.704+0000 7f844ffff640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f843c06af00 0x7f843c06d3c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8444036000 tx=0x7f844403d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.704+0000 7f8454c33640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f843c070d60 0x7f843c073180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.705+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== osd.6 v2:172.21.15.137:6808/4246993982 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f843c073830 con 0x7f843c070d60 2024-01-19T04:57:53.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.718+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 --> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8450156440 con 0x7f843c070d60 2024-01-19T04:57:53.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f844cff9640 1 -- 172.21.15.31:0/2730178583 <== osd.6 v2:172.21.15.137:6808/4246993982 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8450156440 con 0x7f843c070d60 2024-01-19T04:57:53.718 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:53.718 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:53.719 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f843c070d60 msgr2=0x7f843c073180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6808/4246993982,v1:172.21.15.137:6810/4246993982] conn(0x7f843c070d60 0x7f843c073180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f843c06af00 msgr2=0x7f843c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f843c06af00 0x7f843c06d3c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8444036000 tx=0x7f844403d040 comp rx=0 tx=0).stop 2024-01-19T04:57:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc4e0 msgr2=0x7f84501f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 --2- 172.21.15.31:0/2730178583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84500cc4e0 0x7f84501f4390 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f844000c430 tx=0x7f844000c8f0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8454c33640 1 -- 172.21.15.31:0/2730178583 reap_dead start 2024-01-19T04:57:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 shutdown_connections 2024-01-19T04:57:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 >> 172.21.15.31:0/2730178583 conn(0x7f84500b1990 msgr2=0x7f845006cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 shutdown_connections 2024-01-19T04:57:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.719+0000 7f8456696640 1 -- 172.21.15.31:0/2730178583 wait complete. 2024-01-19T04:57:53.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-19T04:57:53.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.831+0000 7f3b145ec640 1 Processor -- start 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.831+0000 7f3b145ec640 1 -- start start 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.832+0000 7f3b145ec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3b0c0716f0 0x7f3b0c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.832+0000 7f3b145ec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c071c20 con 0x7f3b0c0cb2e0 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.832+0000 7f3b145ec640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c071f00 con 0x7f3b0c15b510 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.832+0000 7f3b145ec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0721e0 con 0x7f3b0c0716f0 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.832+0000 7f3b11b87640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3b0c0716f0 0x7f3b0c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.832+0000 7f3b12388640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3b0c0cb2e0 0x7f3b0c15e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38384/0 (socket says 172.21.15.31:38384) 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.832+0000 7f3b12388640 1 -- 172.21.15.31:0/1467701235 learned_addr learned my addr 172.21.15.31:0/1467701235 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:57:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.833+0000 7f3b11b87640 1 -- 172.21.15.31:0/1467701235 >> v1:172.21.15.137:6789/0 conn(0x7f3b0c15b510 legacy=0x7f3b0c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.833+0000 7f3b11b87640 1 -- 172.21.15.31:0/1467701235 >> v1:172.21.15.31:6789/0 conn(0x7f3b0c0cb2e0 legacy=0x7f3b0c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.833+0000 7f3b11b87640 1 -- 172.21.15.31:0/1467701235 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c15aab0 con 0x7f3b0c0716f0 2024-01-19T04:57:53.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.833+0000 7f3b11b87640 1 --2- 172.21.15.31:0/1467701235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c15afd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3b000097a0 tx=0x7f3b0002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3bcba6ce80c6d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.833+0000 7f3b11386640 1 -- 172.21.15.31:0/1467701235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00030610 con 0x7f3b0c0716f0 2024-01-19T04:57:53.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.833+0000 7f3b11386640 1 -- 172.21.15.31:0/1467701235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b00030770 con 0x7f3b0c0716f0 2024-01-19T04:57:53.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.833+0000 7f3b11386640 1 -- 172.21.15.31:0/1467701235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00030b00 con 0x7f3b0c0716f0 2024-01-19T04:57:53.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.834+0000 7f3b145ec640 1 -- 172.21.15.31:0/1467701235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c0716f0 msgr2=0x7f3b0c15afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.834+0000 7f3b145ec640 1 --2- 172.21.15.31:0/1467701235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c15afd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3b000097a0 tx=0x7f3b0002b960 comp rx=0 tx=0).stop 2024-01-19T04:57:53.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.834+0000 7f3b145ec640 1 -- 172.21.15.31:0/1467701235 shutdown_connections 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.834+0000 7f3b145ec640 1 --2- 172.21.15.31:0/1467701235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c15afd0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.834+0000 7f3b145ec640 1 -- 172.21.15.31:0/1467701235 >> 172.21.15.31:0/1467701235 conn(0x7f3b0c06aa00 msgr2=0x7f3b0c06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.834+0000 7f3b145ec640 1 -- 172.21.15.31:0/1467701235 shutdown_connections 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.834+0000 7f3b145ec640 1 -- 172.21.15.31:0/1467701235 wait complete. 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.835+0000 7f3b145ec640 1 Processor -- start 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.835+0000 7f3b145ec640 1 -- start start 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b145ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c13c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b145ec640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3b0c0cb2e0 0x7f3b0c1f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b145ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c15b510 0x7f3b0c151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b145ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c13cd80 con 0x7f3b0c15b510 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b145ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c1511a0 con 0x7f3b0c0716f0 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b145ec640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c150590 con 0x7f3b0c0cb2e0 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c15b510 0x7f3b0c151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c15b510 0x7f3b0c151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58270/0 (socket says 172.21.15.31:58270) 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 -- 172.21.15.31:0/2152143596 learned_addr learned my addr 172.21.15.31:0/2152143596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12388640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c13c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b11b87640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3b0c0cb2e0 0x7f3b0c1f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 -- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3b0c0cb2e0 msgr2=0x7f3b0c1f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3b0c0cb2e0 0x7f3b0c1f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 -- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c0716f0 msgr2=0x7f3b0c13c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c13c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12b89640 1 -- 172.21.15.31:0/2152143596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00009400 con 0x7f3b0c15b510 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b11b87640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3b0c0cb2e0 0x7f3b0c1f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.836+0000 7f3b12388640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c13c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.837+0000 7f3b12b89640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c15b510 0x7f3b0c151630 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c6b0 tx=0x7f3afc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.837+0000 7f3b0affd640 1 -- 172.21.15.31:0/2152143596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc003b70 con 0x7f3b0c15b510 2024-01-19T04:57:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.837+0000 7f3b0affd640 1 -- 172.21.15.31:0/2152143596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc003cf0 con 0x7f3b0c15b510 2024-01-19T04:57:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.837+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c150870 con 0x7f3b0c15b510 2024-01-19T04:57:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.837+0000 7f3b0affd640 1 -- 172.21.15.31:0/2152143596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc003ea0 con 0x7f3b0c15b510 2024-01-19T04:57:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.837+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c150e30 con 0x7f3b0c15b510 2024-01-19T04:57:53.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.838+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c074900 con 0x7f3b0c15b510 2024-01-19T04:57:53.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.838+0000 7f3b0affd640 1 -- 172.21.15.31:0/2152143596 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f3afc005310 con 0x7f3b0c15b510 2024-01-19T04:57:53.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.839+0000 7f3b0affd640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3af806aed0 0x7f3af806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:53.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.839+0000 7f3b0affd640 1 -- 172.21.15.31:0/2152143596 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3afc0890b0 con 0x7f3b0c15b510 2024-01-19T04:57:53.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.839+0000 7f3b12388640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3af806aed0 0x7f3af806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:53.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.840+0000 7f3b12388640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3af806aed0 0x7f3af806d390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005880 tx=0x7f3b04009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:53.843+0000 7f3b0affd640 1 -- 172.21.15.31:0/2152143596 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3afc054280 con 0x7f3b0c15b510 2024-01-19T04:57:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.022+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b0c06ddc0 con 0x7f3af806aed0 2024-01-19T04:57:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.024+0000 7f3b0affd640 1 -- 172.21.15.31:0/2152143596 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3b04001400 con 0x7f3af806aed0 2024-01-19T04:57:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2024-01-19T04:57:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3af806aed0 msgr2=0x7f3af806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3af806aed0 0x7f3af806d390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005880 tx=0x7f3b04009040 comp rx=0 tx=0).stop 2024-01-19T04:57:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c15b510 msgr2=0x7f3b0c151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c15b510 0x7f3b0c151630 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c6b0 tx=0x7f3afc00cb70 comp rx=0 tx=0).stop 2024-01-19T04:57:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 shutdown_connections 2024-01-19T04:57:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f3af806aed0 0x7f3af806d390 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b0c15b510 0x7f3b0c151630 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3b0c0cb2e0 0x7f3b0c1f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 --2- 172.21.15.31:0/2152143596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c0716f0 0x7f3b0c13c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 >> 172.21.15.31:0/2152143596 conn(0x7f3b0c06aa00 msgr2=0x7f3b0c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 shutdown_connections 2024-01-19T04:57:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.025+0000 7f3b145ec640 1 -- 172.21.15.31:0/2152143596 wait complete. 2024-01-19T04:57:54.035 INFO:tasks.ceph:Scrubbing osd.7 2024-01-19T04:57:54.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-19T04:57:54.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.144+0000 7ff01603a640 1 Processor -- start 2024-01-19T04:57:54.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.144+0000 7ff01603a640 1 -- start start 2024-01-19T04:57:54.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.145+0000 7ff01603a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d3250 0x7ff0100d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.145+0000 7ff01603a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff0100d1720 0x7ff0100d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.145+0000 7ff01603a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0100cfd30 con 0x7ff01015d640 2024-01-19T04:57:54.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.145+0000 7ff01603a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100d09b0 con 0x7ff0100d1720 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.145+0000 7ff01603a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100b92f0 con 0x7ff0100d3250 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00ffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff0100d1720 0x7ff0100d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00effd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d3250 0x7ff0100d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00ffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff0100d1720 0x7ff0100d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.31:44198/0 (socket says 172.21.15.31:44198) 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00ffff640 1 -- 172.21.15.31:0/3635079741 learned_addr learned my addr 172.21.15.31:0/3635079741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00effd640 1 -- 172.21.15.31:0/3635079741 >> v2:172.21.15.137:3300/0 conn(0x7ff0100d1720 msgr2=0x7ff0100d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00effd640 1 --2- 172.21.15.31:0/3635079741 >> v2:172.21.15.137:3300/0 conn(0x7ff0100d1720 0x7ff0100d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00effd640 1 -- 172.21.15.31:0/3635079741 >> v1:172.21.15.137:6789/0 conn(0x7ff01015d640 legacy=0x7ff0100b8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00effd640 1 -- 172.21.15.31:0/3635079741 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100d0010 con 0x7ff0100d3250 2024-01-19T04:57:54.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.146+0000 7ff00ffff640 1 --2- 172.21.15.31:0/3635079741 >> v2:172.21.15.137:3300/0 conn(0x7ff0100d1720 0x7ff0100d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-19T04:57:54.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff00effd640 1 --2- 172.21.15.31:0/3635079741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d3250 0x7ff0100d36d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0040097a0 tx=0x7ff00402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a295c085a0da281 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff00e7fc640 1 -- 172.21.15.31:0/3635079741 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004031830 con 0x7ff0100d3250 2024-01-19T04:57:54.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff00e7fc640 1 -- 172.21.15.31:0/3635079741 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff004031990 con 0x7ff0100d3250 2024-01-19T04:57:54.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff00e7fc640 1 -- 172.21.15.31:0/3635079741 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004031d20 con 0x7ff0100d3250 2024-01-19T04:57:54.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff01603a640 1 -- 172.21.15.31:0/3635079741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d3250 msgr2=0x7ff0100d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff01603a640 1 --2- 172.21.15.31:0/3635079741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d3250 0x7ff0100d36d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0040097a0 tx=0x7ff00402b8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff01603a640 1 -- 172.21.15.31:0/3635079741 shutdown_connections 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff01603a640 1 --2- 172.21.15.31:0/3635079741 >> v2:172.21.15.137:3300/0 conn(0x7ff0100d1720 0x7ff0100d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff01603a640 1 --2- 172.21.15.31:0/3635079741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d3250 0x7ff0100d36d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.147+0000 7ff01603a640 1 -- 172.21.15.31:0/3635079741 >> 172.21.15.31:0/3635079741 conn(0x7ff010072310 msgr2=0x7ff01006b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.148+0000 7ff01603a640 1 -- 172.21.15.31:0/3635079741 shutdown_connections 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.148+0000 7ff01603a640 1 -- 172.21.15.31:0/3635079741 wait complete. 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 Processor -- start 2024-01-19T04:57:54.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 -- start start 2024-01-19T04:57:54.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d1720 0x7ff0100c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0100d3250 0x7ff0100b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff01015d640 0x7ff0100b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100c7550 con 0x7ff01015d640 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100c9880 con 0x7ff0100d3250 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.149+0000 7ff01603a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100c6e90 con 0x7ff0100d1720 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff01015d640 0x7ff0100b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff01015d640 0x7ff0100b7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58274/0 (socket says 172.21.15.31:58274) 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 -- 172.21.15.31:0/793089348 learned_addr learned my addr 172.21.15.31:0/793089348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00f7fe640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d1720 0x7ff0100c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00effd640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0100d3250 0x7ff0100b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 -- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d1720 msgr2=0x7ff0100c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff0100d1720 0x7ff0100c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 -- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0100d3250 msgr2=0x7ff0100b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0100d3250 0x7ff0100b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 -- 172.21.15.31:0/793089348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008009b30 con 0x7ff01015d640 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff00ffff640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff01015d640 0x7ff0100b7eb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff00800efc0 tx=0x7ff00800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008018070 con 0x7ff01015d640 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00800f040 con 0x7ff01015d640 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff004009400 con 0x7ff01015d640 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800ce00 con 0x7ff01015d640 2024-01-19T04:57:54.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.150+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0100c6a40 con 0x7ff01015d640 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.151+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0100d05d0 con 0x7ff01015d640 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ff008014480 con 0x7ff01015d640 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7feff7fff640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7feffc06aed0 0x7feffc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff0080894e0 con 0x7ff01015d640 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7feff7fff640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7feffc070d30 0x7feffc073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 --> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feffc073800 con 0x7feffc070d30 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff00808bba0 con 0x7ff01015d640 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7ff00f7fe640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7feffc06aed0 0x7feffc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.152+0000 7ff00effd640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7feffc070d30 0x7feffc073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.153+0000 7ff00f7fe640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7feffc06aed0 0x7feffc06d390 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff0000016e0 tx=0x7ff000009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.153+0000 7ff00effd640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7feffc070d30 0x7feffc073150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.154+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== osd.7 v2:172.21.15.137:6800/1281133662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7feffc073800 con 0x7feffc070d30 2024-01-19T04:57:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.167+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 --> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff0100696e0 con 0x7feffc070d30 2024-01-19T04:57:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.167+0000 7feff7fff640 1 -- 172.21.15.31:0/793089348 <== osd.7 v2:172.21.15.137:6800/1281133662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff0100696e0 con 0x7feffc070d30 2024-01-19T04:57:54.167 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-01-19T04:57:54.167 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-19T04:57:54.167 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-01-19T04:57:54.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7feffc070d30 msgr2=0x7feffc073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6800/1281133662,v1:172.21.15.137:6801/1281133662] conn(0x7feffc070d30 0x7feffc073150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7feffc06aed0 msgr2=0x7feffc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7feffc06aed0 0x7feffc06d390 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff0000016e0 tx=0x7ff000009040 comp rx=0 tx=0).stop 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff01015d640 msgr2=0x7ff0100b7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 --2- 172.21.15.31:0/793089348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff01015d640 0x7ff0100b7eb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff00800efc0 tx=0x7ff00800c490 comp rx=0 tx=0).stop 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff00ffff640 1 -- 172.21.15.31:0/793089348 reap_dead start 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 shutdown_connections 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 >> 172.21.15.31:0/793089348 conn(0x7ff010072310 msgr2=0x7ff0100b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 shutdown_connections 2024-01-19T04:57:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.168+0000 7ff01603a640 1 -- 172.21.15.31:0/793089348 wait complete. 2024-01-19T04:57:54.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-19T04:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.280+0000 7f7c65bde640 1 Processor -- start 2024-01-19T04:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.280+0000 7f7c65bde640 1 -- start start 2024-01-19T04:57:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.280+0000 7f7c65bde640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7c6015d340 0x7f7c600cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.280+0000 7f7c65bde640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c6006acc0 0x7f7c60071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.280+0000 7f7c65bde640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c600c6130 con 0x7f7c600cc490 2024-01-19T04:57:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.280+0000 7f7c65bde640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c600ce630 con 0x7f7c6006acc0 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.280+0000 7f7c65bde640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c600c5520 con 0x7f7c6015d340 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c6006acc0 0x7f7c60071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c6006acc0 0x7f7c60071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58280/0 (socket says 172.21.15.31:58280) 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5ffff640 1 -- 172.21.15.31:0/322379148 learned_addr learned my addr 172.21.15.31:0/322379148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5f7fe640 1 --2- 172.21.15.31:0/322379148 >> v2:172.21.15.137:3300/0 conn(0x7f7c6015d340 0x7f7c600cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5ffff640 1 -- 172.21.15.31:0/322379148 >> v2:172.21.15.137:3300/0 conn(0x7f7c6015d340 msgr2=0x7f7c600cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5ffff640 1 --2- 172.21.15.31:0/322379148 >> v2:172.21.15.137:3300/0 conn(0x7f7c6015d340 0x7f7c600cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5ffff640 1 -- 172.21.15.31:0/322379148 >> v1:172.21.15.137:6789/0 conn(0x7f7c600cc490 legacy=0x7f7c6006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.281+0000 7f7c5ffff640 1 -- 172.21.15.31:0/322379148 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600c5800 con 0x7f7c6006acc0 2024-01-19T04:57:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c5ffff640 1 --2- 172.21.15.31:0/322379148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6006acc0 0x7f7c60071340 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e50 tx=0x7f7c5802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2424858322e1493d server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c5e7fc640 1 -- 172.21.15.31:0/322379148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5802c960 con 0x7f7c6006acc0 2024-01-19T04:57:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c5e7fc640 1 -- 172.21.15.31:0/322379148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5802cac0 con 0x7f7c6006acc0 2024-01-19T04:57:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c5e7fc640 1 -- 172.21.15.31:0/322379148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5802ce50 con 0x7f7c6006acc0 2024-01-19T04:57:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 -- 172.21.15.31:0/322379148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6006acc0 msgr2=0x7f7c60071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 --2- 172.21.15.31:0/322379148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6006acc0 0x7f7c60071340 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e50 tx=0x7f7c5802bdc0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 -- 172.21.15.31:0/322379148 shutdown_connections 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 --2- 172.21.15.31:0/322379148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6006acc0 0x7f7c60071340 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 --2- 172.21.15.31:0/322379148 >> v2:172.21.15.137:3300/0 conn(0x7f7c6015d340 0x7f7c600cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 -- 172.21.15.31:0/322379148 >> 172.21.15.31:0/322379148 conn(0x7f7c600b1c50 msgr2=0x7f7c600b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 -- 172.21.15.31:0/322379148 shutdown_connections 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.282+0000 7f7c65bde640 1 -- 172.21.15.31:0/322379148 wait complete. 2024-01-19T04:57:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.283+0000 7f7c65bde640 1 Processor -- start 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c65bde640 1 -- start start 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c65bde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c6006acc0 0x7f7c601562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c65bde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c600cc490 0x7f7c600c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c65bde640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c6015d340 0x7f7c600b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c65bde640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600c9290 con 0x7f7c600cc490 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c65bde640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600c8680 con 0x7f7c6006acc0 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c65bde640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600c8960 con 0x7f7c6015d340 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.284+0000 7f7c5effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c600cc490 0x7f7c600c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c600cc490 0x7f7c600c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58282/0 (socket says 172.21.15.31:58282) 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 -- 172.21.15.31:0/29451381 learned_addr learned my addr 172.21.15.31:0/29451381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5f7fe640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c6006acc0 0x7f7c601562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5ffff640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c6015d340 0x7f7c600b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 -- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c6015d340 msgr2=0x7f7c600b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c6015d340 0x7f7c600b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 -- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c6006acc0 msgr2=0x7f7c601562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c6006acc0 0x7f7c601562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 -- 172.21.15.31:0/29451381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c58009b30 con 0x7f7c600cc490 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5ffff640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c6015d340 0x7f7c600b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c5effd640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c600cc490 0x7f7c600c7e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c410 tx=0x7f7c5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c47fff640 1 -- 172.21.15.31:0/29451381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540035f0 con 0x7f7c600cc490 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c47fff640 1 -- 172.21.15.31:0/29451381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c54003750 con 0x7f7c600cc490 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c600c8c40 con 0x7f7c600cc490 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c47fff640 1 -- 172.21.15.31:0/29451381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54003b40 con 0x7f7c600cc490 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.285+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c600c8390 con 0x7f7c600cc490 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.286+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c600c9400 con 0x7f7c600cc490 2024-01-19T04:57:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.287+0000 7f7c47fff640 1 -- 172.21.15.31:0/29451381 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f7c54003ca0 con 0x7f7c600cc490 2024-01-19T04:57:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.287+0000 7f7c47fff640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c4c06aed0 0x7f7c4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.287+0000 7f7c47fff640 1 -- 172.21.15.31:0/29451381 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7c54088960 con 0x7f7c600cc490 2024-01-19T04:57:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.287+0000 7f7c5f7fe640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c4c06aed0 0x7f7c4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.288+0000 7f7c5f7fe640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c4c06aed0 0x7f7c4c06d390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000aa20 tx=0x7f7c50008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.291+0000 7f7c47fff640 1 -- 172.21.15.31:0/29451381 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7c54053be0 con 0x7f7c600cc490 2024-01-19T04:57:54.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.469+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c600c8ad0 con 0x7f7c4c06aed0 2024-01-19T04:57:54.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.470+0000 7f7c47fff640 1 -- 172.21.15.31:0/29451381 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7c600c8ad0 con 0x7f7c4c06aed0 2024-01-19T04:57:54.469 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2024-01-19T04:57:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c4c06aed0 msgr2=0x7f7c4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c4c06aed0 0x7f7c4c06d390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000aa20 tx=0x7f7c50008040 comp rx=0 tx=0).stop 2024-01-19T04:57:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c600cc490 msgr2=0x7f7c600c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c600cc490 0x7f7c600c7e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c410 tx=0x7f7c5400c8d0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 shutdown_connections 2024-01-19T04:57:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f7c4c06aed0 0x7f7c4c06d390 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c6015d340 0x7f7c600b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c600cc490 0x7f7c600c7e50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 --2- 172.21.15.31:0/29451381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c6006acc0 0x7f7c601562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 >> 172.21.15.31:0/29451381 conn(0x7f7c600b1c50 msgr2=0x7f7c600cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:54.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 shutdown_connections 2024-01-19T04:57:54.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.472+0000 7f7c65bde640 1 -- 172.21.15.31:0/29451381 wait complete. 2024-01-19T04:57:54.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:57:54.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.584+0000 7f4e45f2d640 1 Processor -- start 2024-01-19T04:57:54.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e45f2d640 1 -- start start 2024-01-19T04:57:54.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e45f2d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e40160bb0 0x7f4e400b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e45f2d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4e400cfd10 0x7f4e400d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e45f2d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4e400d0ef0 0x7f4e400d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e45f2d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e400d06b0 con 0x7f4e400cfd10 2024-01-19T04:57:54.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e45f2d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e400b66b0 con 0x7f4e400d0ef0 2024-01-19T04:57:54.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e45f2d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400d18c0 con 0x7f4e40160bb0 2024-01-19T04:57:54.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e3effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4e400cfd10 0x7f4e400d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e3effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4e400cfd10 0x7f4e400d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58294/0 (socket says 172.21.15.31:58294) 2024-01-19T04:57:54.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.585+0000 7f4e3effd640 1 -- 172.21.15.31:0/492262128 learned_addr learned my addr 172.21.15.31:0/492262128 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3ffff640 1 --2- 172.21.15.31:0/492262128 >> v2:172.21.15.137:3300/0 conn(0x7f4e400d0ef0 0x7f4e400d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3f7fe640 1 --2- 172.21.15.31:0/492262128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e40160bb0 0x7f4e400b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3effd640 1 -- 172.21.15.31:0/492262128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e40160bb0 msgr2=0x7f4e400b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3effd640 1 --2- 172.21.15.31:0/492262128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e40160bb0 0x7f4e400b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3effd640 1 -- 172.21.15.31:0/492262128 >> v2:172.21.15.137:3300/0 conn(0x7f4e400d0ef0 msgr2=0x7f4e400d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:57:54.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3effd640 1 --2- 172.21.15.31:0/492262128 >> v2:172.21.15.137:3300/0 conn(0x7f4e400d0ef0 0x7f4e400d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3effd640 1 -- 172.21.15.31:0/492262128 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e400d1b90 con 0x7f4e400cfd10 2024-01-19T04:57:54.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.586+0000 7f4e3effd640 1 --2- 172.21.15.31:0/492262128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e400cfd10 0x7f4e400d0170 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4e401623d0 tx=0x7f4e3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53681ed063def5ec server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e3e7fc640 1 -- 172.21.15.31:0/492262128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34031870 con 0x7f4e400cfd10 2024-01-19T04:57:54.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e3e7fc640 1 -- 172.21.15.31:0/492262128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e340319d0 con 0x7f4e400cfd10 2024-01-19T04:57:54.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e3e7fc640 1 -- 172.21.15.31:0/492262128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34031d60 con 0x7f4e400cfd10 2024-01-19T04:57:54.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 -- 172.21.15.31:0/492262128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e400cfd10 msgr2=0x7f4e400d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/492262128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e400cfd10 0x7f4e400d0170 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4e401623d0 tx=0x7f4e3402b910 comp rx=0 tx=0).stop 2024-01-19T04:57:54.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 -- 172.21.15.31:0/492262128 shutdown_connections 2024-01-19T04:57:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/492262128 >> v2:172.21.15.137:3300/0 conn(0x7f4e400d0ef0 0x7f4e400d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/492262128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e400cfd10 0x7f4e400d0170 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/492262128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e40160bb0 0x7f4e400b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 -- 172.21.15.31:0/492262128 >> 172.21.15.31:0/492262128 conn(0x7f4e4009c600 msgr2=0x7f4e400d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:54.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 -- 172.21.15.31:0/492262128 shutdown_connections 2024-01-19T04:57:54.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.587+0000 7f4e45f2d640 1 -- 172.21.15.31:0/492262128 wait complete. 2024-01-19T04:57:54.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.588+0000 7f4e45f2d640 1 Processor -- start 2024-01-19T04:57:54.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e45f2d640 1 -- start start 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e45f2d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e400cfd10 0x7f4e4014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e45f2d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e400d0ef0 0x7f4e400c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e45f2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e40160bb0 0x7f4e400c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e45f2d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400c7860 con 0x7f4e40160bb0 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e45f2d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400c7b40 con 0x7f4e400cfd10 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e45f2d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400c7e20 con 0x7f4e400d0ef0 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e40160bb0 0x7f4e400c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e40160bb0 0x7f4e400c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58306/0 (socket says 172.21.15.31:58306) 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.589+0000 7f4e3ffff640 1 -- 172.21.15.31:0/2787907854 learned_addr learned my addr 172.21.15.31:0/2787907854 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3effd640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e400d0ef0 0x7f4e400c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3f7fe640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e400cfd10 0x7f4e4014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3ffff640 1 -- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e400d0ef0 msgr2=0x7f4e400c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3ffff640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e400d0ef0 0x7f4e400c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3ffff640 1 -- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e400cfd10 msgr2=0x7f4e4014f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3ffff640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e400cfd10 0x7f4e4014f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3ffff640 1 -- 172.21.15.31:0/2787907854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e340094d0 con 0x7f4e40160bb0 2024-01-19T04:57:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3effd640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e400d0ef0 0x7f4e400c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3f7fe640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e400cfd10 0x7f4e4014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e3ffff640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e40160bb0 0x7f4e400c8900 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4e38007d20 tx=0x7f4e3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e27fff640 1 -- 172.21.15.31:0/2787907854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38014420 con 0x7f4e40160bb0 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e27fff640 1 -- 172.21.15.31:0/2787907854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e380145a0 con 0x7f4e40160bb0 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e400c8100 con 0x7f4e40160bb0 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e27fff640 1 -- 172.21.15.31:0/2787907854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38014950 con 0x7f4e40160bb0 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.590+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e400c9b30 con 0x7f4e40160bb0 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.591+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e400c8620 con 0x7f4e40160bb0 2024-01-19T04:57:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.592+0000 7f4e27fff640 1 -- 172.21.15.31:0/2787907854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f4e38014ab0 con 0x7f4e40160bb0 2024-01-19T04:57:54.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.592+0000 7f4e27fff640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e2c06aed0 0x7f4e2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:57:54.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.592+0000 7f4e27fff640 1 -- 172.21.15.31:0/2787907854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4e38089be0 con 0x7f4e40160bb0 2024-01-19T04:57:54.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.592+0000 7f4e3f7fe640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e2c06aed0 0x7f4e2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:57:54.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.593+0000 7f4e3f7fe640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e2c06aed0 0x7f4e2c06d390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4e30006fd0 tx=0x7f4e30008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:57:54.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.596+0000 7f4e27fff640 1 -- 172.21.15.31:0/2787907854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4e400c8620 con 0x7f4e40160bb0 2024-01-19T04:57:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.778+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e400c7cb0 con 0x7f4e2c06aed0 2024-01-19T04:57:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.781+0000 7f4e27fff640 1 -- 172.21.15.31:0/2787907854 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51199 (secure 0 0 0) 0x7f4e400c7cb0 con 0x7f4e2c06aed0 2024-01-19T04:57:54.781 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:57:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:57:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.782+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e2c06aed0 msgr2=0x7f4e2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.782+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e2c06aed0 0x7f4e2c06d390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4e30006fd0 tx=0x7f4e30008040 comp rx=0 tx=0).stop 2024-01-19T04:57:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.782+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e40160bb0 msgr2=0x7f4e400c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:57:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.782+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e40160bb0 0x7f4e400c8900 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4e38007d20 tx=0x7f4e3800e4b0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.782+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 shutdown_connections 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.782+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f4e2c06aed0 0x7f4e2c06d390 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.782+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e40160bb0 0x7f4e400c8900 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.783+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4e400d0ef0 0x7f4e400c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.783+0000 7f4e45f2d640 1 --2- 172.21.15.31:0/2787907854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e400cfd10 0x7f4e4014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.783+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 >> 172.21.15.31:0/2787907854 conn(0x7f4e4009c600 msgr2=0x7f4e400b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.783+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 shutdown_connections 2024-01-19T04:57:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:57:54.783+0000 7f4e45f2d640 1 -- 172.21.15.31:0/2787907854 wait complete. 2024-01-19T04:57:54.792 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-01-19T04:57:53.597553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222332,"kb_used_data":6152,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752388,"statfs":{"total":773094113280,"available":772866445312,"internally_reserved":0,"allocated":6299648,"data_stored":3299704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001655"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-01-19T04:57:45.696950+0000","last_change":"2024-01-19T04:55:07.600407+0000","last_active":"2024-01-19T04:57:45.696950+0000","last_peered":"2024-01-19T04:57:45.696950+0000","last_clean":"2024-01-19T04:57:45.696950+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:57:45.696950+0000","last_undegraded":"2024-01-19T04:57:45.696950+0000","last_fullsized":"2024-01-19T04:57:45.696950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-19T04:55:06.576501+0000","last_clean_scrub_stamp":"2024-01-19T04:55:06.576501+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T07:22:31.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":855239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.435},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73099999999999998,"5min":0.753,"15min":0.753},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68200000000000005,"5min":0.745,"15min":0.745},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.505}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68500000000000005,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33700000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.496},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35699999999999998}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.312,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.69999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.59899999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.504},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68799999999999994,"5min":0.876,"15min":0.876},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.69499999999999995,"5min":0.754,"15min":0.754},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.877,"15min":0.877},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36399999999999999}]},{"osd":5,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Jan 19 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92800000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.504,"15min":0.505},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.68799999999999994}]},{"osd":7,"last update":"Fri Jan 19 04:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.628},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.65600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98299999999999998,"5min":1.103,"15min":1.103},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84799999999999998,"5min":1.2,"15min":1.2},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0049999999999999,"5min":1.321,"15min":1.321},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":1.046,"15min":1.046},"last":0.35399999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.439},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.48499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0309999999999999,"5min":1.258,"15min":1.258},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.42799999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.41799999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.318,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53200000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.88,"15min":0.88},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.755,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80300000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71299999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Jan 19 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81999999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.88900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67700000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.314},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38900000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.32900000000000001}]},{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":1.47,"15min":1.47},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":1.006,"15min":1.006},"last":0.38700000000000001}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90900000000000003,"5min":1.355,"15min":1.355},"last":0.432}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.442},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82999999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81899999999999995,"5min":0.995,"15min":0.995},"last":0.41899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56299999999999994,"5min":1.397,"15min":1.397},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.23799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.26800000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.316},{"interface":"front","average":{"1min":0.373,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":1.423,"15min":1.423},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66700000000000004,"5min":1.446,"15min":1.446},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.432},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.38}]},{"osd":4,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":1.619,"15min":1.619},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":1.585,"15min":1.585},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":1.333,"15min":1.333},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Jan 19 04:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83899999999999997,"5min":1.367,"15min":1.367},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.624},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.44900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":855239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.36899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46400000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.307},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.318}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.42299999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35899999999999999}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81200000000000006,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.044,"15min":1.044},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:57:54.796 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-01-19T04:55:06.576501+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=19, tm_hour=4, tm_min=55, tm_sec=6, tm_wday=4, tm_yday=19, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=19, tm_hour=4, tm_min=57, tm_sec=49, tm_wday=4, tm_yday=19, tm_isdst=0) 2024-01-19T04:57:54.796 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-19T04:58:13.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:58:13.971 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:58:14.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-19T04:58:14.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.912+0000 7f4453815640 1 Processor -- start 2024-01-19T04:58:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.912+0000 7f4453815640 1 -- start start 2024-01-19T04:58:14.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4453815640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f444c0b1f40 0x7f444c15b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4453815640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0b2ee0 con 0x7f444c13df90 2024-01-19T04:58:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4453815640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0cba60 con 0x7f444c06d6b0 2024-01-19T04:58:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4453815640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c15b7a0 con 0x7f444c0b1f40 2024-01-19T04:58:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4451db2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f444c13df90 0x7f444c13e3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39212/0 (socket says 172.21.15.31:39212) 2024-01-19T04:58:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4451db2640 1 -- 172.21.15.31:0/1620228899 learned_addr learned my addr 172.21.15.31:0/1620228899 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:58:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4450db0640 1 --2- 172.21.15.31:0/1620228899 >> v2:172.21.15.137:3300/0 conn(0x7f444c0b1f40 0x7f444c15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.913+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198524200 0 0) 0x7f444c0b2ee0 con 0x7f444c13df90 2024-01-19T04:58:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4438005730 con 0x7f444c13df90 2024-01-19T04:58:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1290438155 0 0) 0x7f4438005730 con 0x7f444c13df90 2024-01-19T04:58:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 >> v2:172.21.15.137:3300/0 conn(0x7f444c0b1f40 msgr2=0x7f444c15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:58:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 --2- 172.21.15.31:0/1620228899 >> v2:172.21.15.137:3300/0 conn(0x7f444c0b1f40 0x7f444c15b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 >> v1:172.21.15.137:6789/0 conn(0x7f444c06d6b0 legacy=0x7f444c06da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0b23d0 con 0x7f444c13df90 2024-01-19T04:58:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7f44480019f0 con 0x7f444c13df90 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 >> v1:172.21.15.31:6789/0 conn(0x7f444c13df90 legacy=0x7f444c13e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4438005b80 0x7f4438005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44380069d0 0x7f4438006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438007390 0x7f443800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443800e300 con 0x7f4438005b80 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443800e470 con 0x7f44380069d0 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443800e630 con 0x7f4438007390 2024-01-19T04:58:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f44515b1640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4438005b80 0x7f4438005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4450db0640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44380069d0 0x7f4438006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4451db2640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438007390 0x7f443800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f44515b1640 1 -- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438007390 msgr2=0x7f443800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f44515b1640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438007390 0x7f443800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f44515b1640 1 -- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44380069d0 msgr2=0x7f4438006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f44515b1640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44380069d0 0x7f4438006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f44515b1640 1 -- 172.21.15.31:0/1620228899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438005730 con 0x7f4438005b80 2024-01-19T04:58:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4451db2640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438007390 0x7f443800db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.914+0000 7f4451db2640 1 -- 172.21.15.31:0/1620228899 reap_dead start 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.915+0000 7f44515b1640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4438005b80 0x7f4438005fa0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4444001750 tx=0x7f4438007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689805dc06f2f920 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.915+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440052d0 con 0x7f4438005b80 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.915+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444402e050 con 0x7f4438005b80 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.915+0000 7f4443fff640 1 -- 172.21.15.31:0/1620228899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444003830 con 0x7f4438005b80 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.916+0000 7f4453815640 1 -- 172.21.15.31:0/1620228899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4438005b80 msgr2=0x7f4438005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.916+0000 7f4453815640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4438005b80 0x7f4438005fa0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4444001750 tx=0x7f4438007a00 comp rx=0 tx=0).stop 2024-01-19T04:58:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.916+0000 7f4453815640 1 -- 172.21.15.31:0/1620228899 shutdown_connections 2024-01-19T04:58:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.916+0000 7f4453815640 1 --2- 172.21.15.31:0/1620228899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4438005b80 0x7f4438005fa0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.916+0000 7f4453815640 1 -- 172.21.15.31:0/1620228899 >> 172.21.15.31:0/1620228899 conn(0x7f444c06a790 msgr2=0x7f444c06aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:58:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.916+0000 7f4453815640 1 -- 172.21.15.31:0/1620228899 shutdown_connections 2024-01-19T04:58:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.916+0000 7f4453815640 1 -- 172.21.15.31:0/1620228899 wait complete. 2024-01-19T04:58:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.917+0000 7f4453815640 1 Processor -- start 2024-01-19T04:58:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.917+0000 7f4453815640 1 -- start start 2024-01-19T04:58:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4453815640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438005b80 0x7f444c1ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4453815640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c13bc10 0x7f444c13e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4453815640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f444c13e8e0 0x7f444c0a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4453815640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c13ed90 con 0x7f444c13bc10 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4453815640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c142170 con 0x7f444c13e8e0 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4453815640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c142450 con 0x7f4438005b80 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c13bc10 0x7f444c13e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c13bc10 0x7f444c13e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59010/0 (socket says 172.21.15.31:59010) 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 -- 172.21.15.31:0/3608039763 learned_addr learned my addr 172.21.15.31:0/3608039763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f44515b1640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438005b80 0x7f444c1ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4451db2640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f444c13e8e0 0x7f444c0a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 -- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438005b80 msgr2=0x7f444c1ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:14.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438005b80 0x7f444c1ef9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 -- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f444c13e8e0 msgr2=0x7f444c0a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f444c13e8e0 0x7f444c0a8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4450db0640 1 -- 172.21.15.31:0/3608039763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c142730 con 0x7f444c13bc10 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.918+0000 7f4451db2640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f444c13e8e0 0x7f444c0a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.919+0000 7f4450db0640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c13bc10 0x7f444c13e3a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f443c011a70 tx=0x7f443c011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.919+0000 7f4441ffb640 1 -- 172.21.15.31:0/3608039763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c009680 con 0x7f444c13bc10 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.919+0000 7f4441ffb640 1 -- 172.21.15.31:0/3608039763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c00fb50 con 0x7f444c13bc10 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.919+0000 7f4441ffb640 1 -- 172.21.15.31:0/3608039763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c00fee0 con 0x7f444c13bc10 2024-01-19T04:58:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.919+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c142a10 con 0x7f444c13bc10 2024-01-19T04:58:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.919+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c13c090 con 0x7f444c13bc10 2024-01-19T04:58:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.920+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c0696e0 con 0x7f444c13bc10 2024-01-19T04:58:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.921+0000 7f4441ffb640 1 -- 172.21.15.31:0/3608039763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7f443c014440 con 0x7f444c13bc10 2024-01-19T04:58:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.921+0000 7f4441ffb640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f443006af00 0x7f443006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.921+0000 7f4441ffb640 1 -- 172.21.15.31:0/3608039763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f443c089a90 con 0x7f444c13bc10 2024-01-19T04:58:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.921+0000 7f44515b1640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f443006af00 0x7f443006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.922+0000 7f44515b1640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f443006af00 0x7f443006d3c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4444033000 tx=0x7f4444009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:14.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:14.925+0000 7f4441ffb640 1 -- 172.21.15.31:0/3608039763 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f443c054cf0 con 0x7f444c13bc10 2024-01-19T04:58:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.102+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 --> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f444c0b2c00 con 0x7f443006af00 2024-01-19T04:58:15.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.104+0000 7f4441ffb640 1 -- 172.21.15.31:0/3608039763 <== mgr.4112 v2:172.21.15.137:6832/4063332792 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51149 (secure 0 0 0) 0x7f44440053d0 con 0x7f443006af00 2024-01-19T04:58:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:58:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-19T04:58:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f443006af00 msgr2=0x7f443006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f443006af00 0x7f443006d3c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4444033000 tx=0x7f4444009030 comp rx=0 tx=0).stop 2024-01-19T04:58:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c13bc10 msgr2=0x7f444c13e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c13bc10 0x7f444c13e3a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f443c011a70 tx=0x7f443c011f30 comp rx=0 tx=0).stop 2024-01-19T04:58:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 shutdown_connections 2024-01-19T04:58:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7f443006af00 0x7f443006d3c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f444c13e8e0 0x7f444c0a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c13bc10 0x7f444c13e3a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 --2- 172.21.15.31:0/3608039763 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4438005b80 0x7f444c1ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.106+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 >> 172.21.15.31:0/3608039763 conn(0x7f444c06a790 msgr2=0x7f444c0af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:58:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.107+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 shutdown_connections 2024-01-19T04:58:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.107+0000 7f4453815640 1 -- 172.21.15.31:0/3608039763 wait complete. 2024-01-19T04:58:15.117 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-01-19T04:58:13.601258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222324,"kb_used_data":6152,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752396,"statfs":{"total":773094113280,"available":772866453504,"internally_reserved":0,"allocated":6299648,"data_stored":3299704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002333"},"pg_stats":[{"pgid":"1.0","version":"53'119","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-19T04:57:54.825411+0000","last_change":"2024-01-19T04:57:54.825449+0000","last_active":"2024-01-19T04:57:54.825411+0000","last_peered":"2024-01-19T04:57:54.825411+0000","last_clean":"2024-01-19T04:57:54.825411+0000","last_became_active":"2024-01-19T04:55:07.599499+0000","last_became_peered":"2024-01-19T04:55:07.599499+0000","last_unstale":"2024-01-19T04:57:54.825411+0000","last_undegraded":"2024-01-19T04:57:54.825411+0000","last_fullsized":"2024-01-19T04:57:54.825411+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2024-01-19T04:57:54.825326+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2024-01-19T04:57:54.825326+0000","last_clean_scrub_stamp":"2024-01-19T04:57:54.825326+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-20T09:53:55.760802+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":855239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73399999999999999,"5min":0.753,"15min":0.753},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Jan 19 04:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Jan 19 04:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.66000000000000003,"15min":0.69999999999999996},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.748,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.44600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Jan 19 04:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.504},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68899999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Jan 19 04:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.625,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69299999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38300000000000001}]},{"osd":5,"last update":"Fri Jan 19 04:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77000000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.69799999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.45500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.77500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.39200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.45900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.504}]},{"osd":2,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.55200000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.876,"15min":0.876},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.75800000000000001,"5min":0.877,"15min":0.877},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44600000000000001}]},{"osd":5,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67900000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.35399999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72599999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74099999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.67100000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76200000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.35599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68999999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65600000000000003,"5min":0.875,"15min":0.875},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71399999999999997,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.47299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52700000000000002,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73799999999999999,"5min":1,"15min":1},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.502,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.39900000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.33300000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68200000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.626}]},{"osd":7,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.69999999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72599999999999998,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82299999999999995,"5min":1.103,"15min":1.103},"last":0.621},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71799999999999997,"5min":1.2,"15min":1.2},"last":0.71799999999999997}]},{"osd":1,"last update":"Fri Jan 19 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84499999999999997,"5min":1.321,"15min":1.321},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68500000000000005,"5min":1.046,"15min":1.046},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Jan 19 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.755,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Jan 19 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0700000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.052,"5min":1.258,"15min":1.258},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jan 19 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.87,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Jan 19 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Jan 19 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77300000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.5},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61199999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85899999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32300000000000001,"15min":0.32100000000000001},"min":{"1min":0.221,"5min":0.184,"15min":0.184},"max":{"1min":0.48899999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.23400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54900000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35199999999999998,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.52400000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.26000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.58099999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.496}]},{"osd":2,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.40300000000000002},"min":{"1min":0.27300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56699999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.41599999999999998},"min":{"1min":0.28299999999999997,"5min":0.219,"15min":0.219},"max":{"1min":0.66400000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.56799999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76900000000000002,"5min":0.88,"15min":0.88},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.439,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84499999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.752},{"interface":"front","average":{"1min":0.623,"5min":0.625,"15min":0.64300000000000002},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.95199999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.95199999999999996}]},{"osd":7,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81699999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89700000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.89700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35199999999999998,"15min":0.35799999999999998},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.43099999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38100000000000001,"15min":0.39500000000000002},"min":{"1min":0.25800000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48099999999999998,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.29599999999999999}]},{"osd":1,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.38600000000000001,"15min":0.40100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52100000000000002,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42399999999999999,"15min":0.438},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48199999999999998,"5min":1.47,"15min":1.47},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41199999999999998,"15min":0.41899999999999998},"min":{"1min":0.28799999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.50600000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.53600000000000003,"5min":1.006,"15min":1.006},"last":0.36499999999999999}]},{"osd":4,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77700000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.432,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.71799999999999997,"5min":1.355,"15min":1.355},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.63100000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78300000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80200000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.72899999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Jan 19 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65100000000000002,"15min":0.67100000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80200000000000005,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78800000000000003,"5min":0.995,"15min":0.995},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":264871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 19 04:58:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35999999999999999,"15min":0.37},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":1.397,"15min":1.397},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34000000000000002},"min":{"1min":0.25700000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.57899999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Jan 19 04:58:11 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.76200000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.308},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53600000000000003,"5min":1.423,"15min":1.423},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Jan 19 04:58:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.433},"min":{"1min":0.34699999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58799999999999997,"5min":1.446,"15min":1.446},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41499999999999998,"15min":0.42899999999999999},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61399999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Jan 19 04:58:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56100000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":1.619,"15min":1.619},"last":0.625}]},{"osd":5,"last update":"Fri Jan 19 04:58:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.755,"5min":1.585,"15min":1.585},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81599999999999995,"5min":1.333,"15min":1.333},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Jan 19 04:58:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85799999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89500000000000002,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Jan 19 04:58:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84799999999999998,"5min":1.367,"15min":1.367},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.44900000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.877,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":855239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.36899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46400000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.42299999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.877}]},{"osd":5,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.747}]},{"osd":6,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81200000000000006,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.85999999999999999}]},{"osd":7,"last update":"Fri Jan 19 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.044,"15min":1.044},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-19T04:58:15.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-19T04:58:15.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.225+0000 7ffa48d1f640 1 Processor -- start 2024-01-19T04:58:15.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.225+0000 7ffa48d1f640 1 -- start start 2024-01-19T04:58:15.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.225+0000 7ffa48d1f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ffa44161860 0x7ffa44168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.225+0000 7ffa48d1f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa440cf9f0 con 0x7ffa440cf570 2024-01-19T04:58:15.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.225+0000 7ffa48d1f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa440cfcd0 con 0x7ffa4415d3a0 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.225+0000 7ffa48d1f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa440cffb0 con 0x7ffa44161860 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa3bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffa440cf570 0x7ffa44161150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39220/0 (socket says 172.21.15.31:39220) 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa3bfff640 1 -- 172.21.15.31:0/2497337524 learned_addr learned my addr 172.21.15.31:0/2497337524 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa42d9d640 1 --2- 172.21.15.31:0/2497337524 >> v2:172.21.15.137:3300/0 conn(0x7ffa44161860 0x7ffa44168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485893378 0 0) 0x7ffa440cf9f0 con 0x7ffa440cf570 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa30003520 con 0x7ffa440cf570 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3335910847 0 0) 0x7ffa30003520 con 0x7ffa440cf570 2024-01-19T04:58:15.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 >> v2:172.21.15.137:3300/0 conn(0x7ffa44161860 msgr2=0x7ffa44168490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 --2- 172.21.15.31:0/2497337524 >> v2:172.21.15.137:3300/0 conn(0x7ffa44161860 0x7ffa44168490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 >> v1:172.21.15.137:6789/0 conn(0x7ffa4415d3a0 legacy=0x7ffa4406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa440d0290 con 0x7ffa440cf570 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 700326683 0 0) 0x7ffa2c001670 con 0x7ffa440cf570 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 >> v1:172.21.15.31:6789/0 conn(0x7ffa440cf570 legacy=0x7ffa44161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa30003970 0x7ffa30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa300047c0 0x7ffa30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30005180 0x7ffa3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440cffb0 con 0x7ffa30003970 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3000c120 con 0x7ffa300047c0 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3000c2e0 con 0x7ffa30005180 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa4259c640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa30003970 0x7ffa30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa3bfff640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa300047c0 0x7ffa30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.226+0000 7ffa42d9d640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30005180 0x7ffa3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa4259c640 1 -- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30005180 msgr2=0x7ffa3000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa4259c640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30005180 0x7ffa3000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa4259c640 1 -- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa300047c0 msgr2=0x7ffa30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa4259c640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa300047c0 0x7ffa30004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa4259c640 1 -- 172.21.15.31:0/2497337524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa30003520 con 0x7ffa30003970 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa3bfff640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa300047c0 0x7ffa30004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa42d9d640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30005180 0x7ffa3000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa42d9d640 1 -- 172.21.15.31:0/2497337524 reap_dead start 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa4259c640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa30003970 0x7ffa30003d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ffa34001770 tx=0x7ffa300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40610b3a44f743c server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.227+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa340052d0 con 0x7ffa30003970 2024-01-19T04:58:15.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3402e050 con 0x7ffa30003970 2024-01-19T04:58:15.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa41d9b640 1 -- 172.21.15.31:0/2497337524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34003640 con 0x7ffa30003970 2024-01-19T04:58:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa48d1f640 1 -- 172.21.15.31:0/2497337524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa30003970 msgr2=0x7ffa30003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa30003970 0x7ffa30003d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ffa34001770 tx=0x7ffa300057f0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa48d1f640 1 -- 172.21.15.31:0/2497337524 shutdown_connections 2024-01-19T04:58:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/2497337524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa30003970 0x7ffa30003d90 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa48d1f640 1 -- 172.21.15.31:0/2497337524 >> 172.21.15.31:0/2497337524 conn(0x7ffa440d1100 msgr2=0x7ffa440d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:58:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa48d1f640 1 -- 172.21.15.31:0/2497337524 shutdown_connections 2024-01-19T04:58:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.228+0000 7ffa48d1f640 1 -- 172.21.15.31:0/2497337524 wait complete. 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.229+0000 7ffa48d1f640 1 Processor -- start 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.229+0000 7ffa48d1f640 1 -- start start 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa48d1f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30003970 0x7ffa441f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa48d1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa441e6990 0x7ffa44157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa48d1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa440ae340 0x7ffa440ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa48d1f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa441f4410 con 0x7ffa440ae340 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa48d1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440aed00 con 0x7ffa441e6990 2024-01-19T04:58:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa48d1f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa441e6e10 con 0x7ffa30003970 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa440ae340 0x7ffa440ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa440ae340 0x7ffa440ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59032/0 (socket says 172.21.15.31:59032) 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 -- 172.21.15.31:0/4071013324 learned_addr learned my addr 172.21.15.31:0/4071013324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa4259c640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30003970 0x7ffa441f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa3bfff640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa441e6990 0x7ffa44157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 -- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30003970 msgr2=0x7ffa441f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30003970 0x7ffa441f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 -- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa441e6990 msgr2=0x7ffa44157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa441e6990 0x7ffa44157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.230+0000 7ffa42d9d640 1 -- 172.21.15.31:0/4071013324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa441e70c0 con 0x7ffa440ae340 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.231+0000 7ffa42d9d640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa440ae340 0x7ffa440ae7c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b7180 tx=0x7ffa3c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.231+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c01b070 con 0x7ffa440ae340 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.231+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c00f040 con 0x7ffa440ae340 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.231+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00e050 con 0x7ffa440ae340 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.231+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa441e73a0 con 0x7ffa440ae340 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.231+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa441e7960 con 0x7ffa440ae340 2024-01-19T04:58:15.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.232+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa44074900 con 0x7ffa440ae340 2024-01-19T04:58:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.232+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7ffa3c003bd0 con 0x7ffa440ae340 2024-01-19T04:58:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.233+0000 7ffa3affd640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffa2406af00 0x7ffa2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.233+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa3c0896e0 con 0x7ffa440ae340 2024-01-19T04:58:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.233+0000 7ffa4259c640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffa2406af00 0x7ffa2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.233+0000 7ffa4259c640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffa2406af00 0x7ffa2406d3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffa34033000 tx=0x7ffa34009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.237+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ffa3c054940 con 0x7ffa440ae340 2024-01-19T04:58:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.416+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ffa44067030 con 0x7ffa440ae340 2024-01-19T04:58:15.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.419+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7ffa3c057f60 con 0x7ffa440ae340 2024-01-19T04:58:15.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.420+0000 7ffa3affd640 1 -- 172.21.15.31:0/4071013324 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ffa3c0098f0 con 0x7ffa440ae340 2024-01-19T04:58:15.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffa2406af00 msgr2=0x7ffa2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffa2406af00 0x7ffa2406d3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffa34033000 tx=0x7ffa34009030 comp rx=0 tx=0).stop 2024-01-19T04:58:15.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa440ae340 msgr2=0x7ffa440ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa440ae340 0x7ffa440ae7c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b7180 tx=0x7ffa3c010590 comp rx=0 tx=0).stop 2024-01-19T04:58:15.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 shutdown_connections 2024-01-19T04:58:15.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7ffa2406af00 0x7ffa2406d3c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa440ae340 0x7ffa440ae7c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa441e6990 0x7ffa44157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 --2- 172.21.15.31:0/4071013324 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa30003970 0x7ffa441f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 >> 172.21.15.31:0/4071013324 conn(0x7ffa440d1100 msgr2=0x7ffa4415e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:58:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 shutdown_connections 2024-01-19T04:58:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.421+0000 7ffa48d1f640 1 -- 172.21.15.31:0/4071013324 wait complete. 2024-01-19T04:58:15.431 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-19T04:58:15.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-19T04:58:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.534+0000 7efe0a6f3640 1 Processor -- start 2024-01-19T04:58:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.534+0000 7efe0a6f3640 1 -- start start 2024-01-19T04:58:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe0a6f3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040d09f0 0x7efe0406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe0a6f3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe0406aaf0 0x7efe040b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe0a6f3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040d0e70 con 0x7efe04154720 2024-01-19T04:58:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe0a6f3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0406af40 con 0x7efe0406aaf0 2024-01-19T04:58:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe0a6f3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040cff50 con 0x7efe040d09f0 2024-01-19T04:58:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe08c90640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe0406aaf0 0x7efe040b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe03fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efe04154720 0x7efe0415da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39232/0 (socket says 172.21.15.31:39232) 2024-01-19T04:58:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe08c90640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe0406aaf0 0x7efe040b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59042/0 (socket says 172.21.15.31:59042) 2024-01-19T04:58:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.535+0000 7efe03fff640 1 -- 172.21.15.31:0/888771523 learned_addr learned my addr 172.21.15.31:0/888771523 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-19T04:58:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.536+0000 7efe037fe640 1 --2- 172.21.15.31:0/888771523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040d09f0 0x7efe0406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.536+0000 7efe08c90640 1 -- 172.21.15.31:0/888771523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040d09f0 msgr2=0x7efe0406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-19T04:58:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.536+0000 7efe08c90640 1 --2- 172.21.15.31:0/888771523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040d09f0 0x7efe0406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.536+0000 7efe08c90640 1 -- 172.21.15.31:0/888771523 >> v1:172.21.15.31:6789/0 conn(0x7efe04154720 legacy=0x7efe0415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.536+0000 7efe08c90640 1 -- 172.21.15.31:0/888771523 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0415e550 con 0x7efe0406aaf0 2024-01-19T04:58:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.536+0000 7efe08c90640 1 --2- 172.21.15.31:0/888771523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe040b3030 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009d90 tx=0x7efdfc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741f22b3ffc3e363 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe02ffd640 1 -- 172.21.15.31:0/888771523 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc02cc50 con 0x7efe0406aaf0 2024-01-19T04:58:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe02ffd640 1 -- 172.21.15.31:0/888771523 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efdfc02cdb0 con 0x7efe0406aaf0 2024-01-19T04:58:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe02ffd640 1 -- 172.21.15.31:0/888771523 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc034680 con 0x7efe0406aaf0 2024-01-19T04:58:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 -- 172.21.15.31:0/888771523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 msgr2=0x7efe040b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/888771523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe040b3030 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009d90 tx=0x7efdfc02bea0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 -- 172.21.15.31:0/888771523 shutdown_connections 2024-01-19T04:58:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/888771523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe040b3030 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/888771523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe040d09f0 0x7efe0406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 -- 172.21.15.31:0/888771523 >> 172.21.15.31:0/888771523 conn(0x7efe0406edc0 msgr2=0x7efe0406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:58:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 -- 172.21.15.31:0/888771523 shutdown_connections 2024-01-19T04:58:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.537+0000 7efe0a6f3640 1 -- 172.21.15.31:0/888771523 wait complete. 2024-01-19T04:58:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.538+0000 7efe0a6f3640 1 Processor -- start 2024-01-19T04:58:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe0a6f3640 1 -- start start 2024-01-19T04:58:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe0a6f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe0414f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe0a6f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040d09f0 0x7efe041e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe0a6f3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe04154720 0x7efe04152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe0a6f3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041504f0 con 0x7efe0406aaf0 2024-01-19T04:58:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe0a6f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0414f9e0 con 0x7efe040d09f0 2024-01-19T04:58:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe0a6f3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040cef10 con 0x7efe04154720 2024-01-19T04:58:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe03fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe0414f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe03fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe0414f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59052/0 (socket says 172.21.15.31:59052) 2024-01-19T04:58:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe03fff640 1 -- 172.21.15.31:0/1037180367 learned_addr learned my addr 172.21.15.31:0/1037180367 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-19T04:58:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.539+0000 7efe08c90640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe04154720 0x7efe04152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe037fe640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040d09f0 0x7efe041e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe03fff640 1 -- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe04154720 msgr2=0x7efe04152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe03fff640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe04154720 0x7efe04152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe03fff640 1 -- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040d09f0 msgr2=0x7efe041e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe03fff640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040d09f0 0x7efe041e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe03fff640 1 -- 172.21.15.31:0/1037180367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc009a70 con 0x7efe0406aaf0 2024-01-19T04:58:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe03fff640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe0414f4a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c400 tx=0x7efdf800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe00ff9640 1 -- 172.21.15.31:0/1037180367 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8005ce0 con 0x7efe0406aaf0 2024-01-19T04:58:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe00ff9640 1 -- 172.21.15.31:0/1037180367 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efdf8005e40 con 0x7efe0406aaf0 2024-01-19T04:58:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe040cf1f0 con 0x7efe0406aaf0 2024-01-19T04:58:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe00ff9640 1 -- 172.21.15.31:0/1037180367 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80072f0 con 0x7efe0406aaf0 2024-01-19T04:58:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.540+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe040cf7b0 con 0x7efe0406aaf0 2024-01-19T04:58:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.541+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe040696e0 con 0x7efe0406aaf0 2024-01-19T04:58:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.541+0000 7efe00ff9640 1 -- 172.21.15.31:0/1037180367 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89198+0+0 (secure 0 0 0) 0x7efdf8007450 con 0x7efe0406aaf0 2024-01-19T04:58:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.542+0000 7efe00ff9640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006af00 0x7efdf006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-19T04:58:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.542+0000 7efe00ff9640 1 -- 172.21.15.31:0/1037180367 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efdf8088750 con 0x7efe0406aaf0 2024-01-19T04:58:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.542+0000 7efe037fe640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006af00 0x7efdf006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-19T04:58:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.543+0000 7efe037fe640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006af00 0x7efdf006d3c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006fd0 tx=0x7efdf4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-19T04:58:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.546+0000 7efe00ff9640 1 -- 172.21.15.31:0/1037180367 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efdf8053920 con 0x7efe0406aaf0 2024-01-19T04:58:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.759+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe040d0e70 con 0x7efe0406aaf0 2024-01-19T04:58:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.760+0000 7efe00ff9640 1 -- 172.21.15.31:0/1037180367 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3705 (secure 0 0 0) 0x7efe040d0e70 con 0x7efe0406aaf0 2024-01-19T04:58:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-19T04:58:15.760 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6836/3002045021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3002045021},{"type":"v1","addr":"172.21.15.137:6836","nonce":3002045021}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1415057458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6835","nonce":1415057458},{"type":"v1","addr":"172.21.15.137:6837","nonce":1415057458}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5919,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/2062734261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2062734261},{"type":"v1","addr":"172.21.15.31:6833","nonce":2062734261}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2158082666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2158082666},{"type":"v1","addr":"172.21.15.31:6835","nonce":2158082666}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[]} 2024-01-19T04:58:15.760 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2024-01-19T04:58:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.761+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006af00 msgr2=0x7efdf006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.761+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006af00 0x7efdf006d3c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006fd0 tx=0x7efdf4008040 comp rx=0 tx=0).stop 2024-01-19T04:58:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 msgr2=0x7efe0414f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe0414f4a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c400 tx=0x7efdf800c8c0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 shutdown_connections 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:6832/4063332792,v1:172.21.15.137:6833/4063332792] conn(0x7efdf006af00 0x7efdf006d3c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe04154720 0x7efe04152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe040d09f0 0x7efe041e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 --2- 172.21.15.31:0/1037180367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0406aaf0 0x7efe0414f4a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 >> 172.21.15.31:0/1037180367 conn(0x7efe0406edc0 msgr2=0x7efe040b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-19T04:58:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 shutdown_connections 2024-01-19T04:58:15.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-19T04:58:15.762+0000 7efe0a6f3640 1 -- 172.21.15.31:0/1037180367 wait complete. 2024-01-19T04:58:15.772 DEBUG:tasks.cephfs.filesystem:already dead... 2024-01-19T04:58:15.772 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-19T04:58:15.772 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-19T04:58:15.772 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:15.781 INFO:tasks.ceph.mds.a:Stopped 2024-01-19T04:58:15.782 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-19T04:58:15.782 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:15.831 INFO:tasks.ceph.mds.c:Stopped 2024-01-19T04:58:15.832 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-19T04:58:15.832 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:15.844 INFO:tasks.ceph.mds.b:Stopped 2024-01-19T04:58:15.844 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-19T04:58:15.844 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:15.896 INFO:tasks.ceph.mds.d:Stopped 2024-01-19T04:58:15.897 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-19T04:58:15.897 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-19T04:58:15.897 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:15.930 INFO:tasks.ceph.osd.0:Stopped 2024-01-19T04:58:15.930 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-19T04:58:15.931 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.005 INFO:tasks.ceph.osd.1:Stopped 2024-01-19T04:58:16.005 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-19T04:58:16.005 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.077 INFO:tasks.ceph.osd.2:Stopped 2024-01-19T04:58:16.077 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-19T04:58:16.077 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.148 INFO:tasks.ceph.osd.3:Stopped 2024-01-19T04:58:16.149 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-19T04:58:16.149 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.181 INFO:tasks.ceph.osd.4:Stopped 2024-01-19T04:58:16.182 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-19T04:58:16.182 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.258 INFO:tasks.ceph.osd.5:Stopped 2024-01-19T04:58:16.259 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-19T04:58:16.259 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.333 INFO:tasks.ceph.osd.6:Stopped 2024-01-19T04:58:16.334 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-19T04:58:16.334 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.408 INFO:tasks.ceph.osd.7:Stopped 2024-01-19T04:58:16.408 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-19T04:58:16.408 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-19T04:58:16.409 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.432 INFO:tasks.ceph.mgr.y:Stopped 2024-01-19T04:58:16.432 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-19T04:58:16.432 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.476 INFO:tasks.ceph.mgr.x:Stopped 2024-01-19T04:58:16.476 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-19T04:58:16.477 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-19T04:58:16.477 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.487 INFO:tasks.ceph.mon.a:Stopped 2024-01-19T04:58:16.488 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-19T04:58:16.488 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.530 INFO:tasks.ceph.mon.b:Stopped 2024-01-19T04:58:16.530 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-19T04:58:16.530 INFO:teuthology.orchestra.run:waiting for 300 2024-01-19T04:58:16.583 INFO:tasks.ceph.mon.c:Stopped 2024-01-19T04:58:16.583 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-19T04:58:16.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-19T04:58:16.616 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:58:16.616 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-19T04:58:16.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:58:16.884 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-19T04:58:16.968 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:58:16.969 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-19T04:58:17.032 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:58:17.032 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-19T04:58:17.084 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:58:17.085 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-19T04:58:17.243 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:58:17.243 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-19T04:58:17.337 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:58:17.337 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-19T04:58:17.397 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:58:17.397 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-19T04:58:17.453 INFO:tasks.ceph:Archiving mon data... 2024-01-19T04:58:17.455 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/data/mon.a.tgz 2024-01-19T04:58:17.455 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-01-19T04:58:17.470 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.E83FBqPUPo 2024-01-19T04:58:17.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.E83FBqPUPo 2024-01-19T04:58:17.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.E83FBqPUPo 2024-01-19T04:58:17.695 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.E83FBqPUPo is 338KB 2024-01-19T04:58:17.792 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.E83FBqPUPo 2024-01-19T04:58:17.806 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/data/mon.b.tgz 2024-01-19T04:58:17.806 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-01-19T04:58:17.824 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.QJ5RsO31sI 2024-01-19T04:58:17.824 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.QJ5RsO31sI 2024-01-19T04:58:17.985 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.QJ5RsO31sI 2024-01-19T04:58:18.072 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.QJ5RsO31sI is 445KB 2024-01-19T04:58:18.171 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.QJ5RsO31sI 2024-01-19T04:58:18.186 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/data/mon.c.tgz 2024-01-19T04:58:18.186 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-01-19T04:58:18.239 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.G1scLxMwfO 2024-01-19T04:58:18.240 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.G1scLxMwfO 2024-01-19T04:58:18.390 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.G1scLxMwfO 2024-01-19T04:58:18.476 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.G1scLxMwfO is 444KB 2024-01-19T04:58:18.573 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.G1scLxMwfO 2024-01-19T04:58:18.587 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-19T04:58:18.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-19T04:58:18.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-19T04:58:18.677 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-19T04:58:18.708 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-19T04:58:18.709 INFO:tasks.ceph:Archiving crash dumps... 2024-01-19T04:58:18.711 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/remote/smithi031/crash 2024-01-19T04:58:18.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-19T04:58:18.747 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/remote/smithi137/crash 2024-01-19T04:58:18.748 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-19T04:58:18.774 INFO:tasks.ceph:Shutting down logrotate 2024-01-19T04:58:18.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:58:18.793 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-19T04:58:18.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-19T04:58:18.903 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-19T04:58:18.930 INFO:tasks.ceph:Compressing logs... 2024-01-19T04:58:18.930 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-19T04:58:18.946 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-19T04:58:18.972 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42335.log 2024-01-19T04:58:18.972 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41848.log 2024-01-19T04:58:18.972 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-01-19T04:58:18.973 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42335.log.gz 2024-01-19T04:58:18.973 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57618.log 2024-01-19T04:58:19.011 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36904.log 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:gzip/var/log/ceph/ceph-osd.6.log: -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.36904.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph.audit.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36904.log.gz 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43604.log 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36704.log 2024-01-19T04:58:19.012 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.3.43604.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.36704.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36664.log 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43151.log 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.7.log: 90.4% -- replaced with /var/log/ceph/ceph-client.3.43604.log.gz 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr: 93.7%/var/log/ceph/ceph-client.admin.36664.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36864.log 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36704.log.gz 2024-01-19T04:58:19.013 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.1.43151.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36664.log.gz 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5/var/log/ceph/ceph-client.admin.36864.log: --verbose -- /var/log/ceph/ceph-client.admin.36784.log 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mgr.x.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36864.log.gz 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36744.log 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr: -- replaced with /var/log/ceph/ceph-client.1.43151.log.gz 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36784.log: -- /var/log/ceph/ceph.log 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.36744.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36784.log.gz 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36824.log 2024-01-19T04:58:19.014 INFO:teuthology.orchestra.run.smithi137.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36744.log.gz 2024-01-19T04:58:19.015 INFO:teuthology.orchestra.run.smithi137.stderr:gzip/var/log/ceph/ceph.log: -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-19T04:58:19.015 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.36824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36608.log 2024-01-19T04:58:19.015 INFO:teuthology.orchestra.run.smithi137.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36824.log.gz 2024-01-19T04:58:19.015 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-19T04:58:19.019 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-19T04:58:19.019 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.36608.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-19T04:58:19.019 INFO:teuthology.orchestra.run.smithi137.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36608.log.gz 2024-01-19T04:58:19.019 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mds.d.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-19T04:58:19.047 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mon.c.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-19T04:58:19.116 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41848.log.gz 2024-01-19T04:58:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-01-19T04:58:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55436.log 2024-01-19T04:58:19.116 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-01-19T04:58:19.116 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57618.log.gz 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43617.log 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51532.log 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43617.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55436.log.gz 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46972.log 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51532.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43617.log.gz 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43956.log 2024-01-19T04:58:19.117 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51532.log.gz 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42643.log 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43956.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46972.log.gz 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42643.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43956.log.gz 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42643.log.gz/var/log/ceph/ceph-client.admin.51146.log: 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59132.log 2024-01-19T04:58:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.a.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60163.log 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42106.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59132.log.gz 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54362.log 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50536.log 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49724.log 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60163.log.gz 2024-01-19T04:58:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50536.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54362.log.gz 2024-01-19T04:58:19.120 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56740.log 2024-01-19T04:58:19.120 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50536.log.gz 2024-01-19T04:58:19.120 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52530.log 2024-01-19T04:58:19.120 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49724.log: /var/log/ceph/ceph-client.admin.56740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53509.log 2024-01-19T04:58:19.120 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49724.log.gz 2024-01-19T04:58:19.120 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43284.log 2024-01-19T04:58:19.120 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56740.log.gz 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53509.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52530.log.gz 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53509.log.gz 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55228.log 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60121.log 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43284.log.gz 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47884.log 2024-01-19T04:58:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60121.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55228.log.gz 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42599.log 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47884.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60121.log.gz 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44575.log 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47884.log.gz 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47539.log 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44575.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42599.log.gz 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50282.log 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47539.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44575.log.gz 2024-01-19T04:58:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47293.log 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47539.log.gz 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50410.log 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43620.log 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50282.log.gz 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50410.log: gzip 89.1% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47293.log.gz /var/log/ceph/ceph-client.admin.60499.log 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50410.log.gz 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54927.log 2024-01-19T04:58:19.123 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43620.log.gz 2024-01-19T04:58:19.124 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-01-19T04:58:19.124 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54927.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60499.log.gz 2024-01-19T04:58:19.124 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52960.log 2024-01-19T04:58:19.124 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54927.log.gz 2024-01-19T04:58:19.124 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53777.log 2024-01-19T04:58:19.124 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-01-19T04:58:19.124 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55762.log 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58964.log 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52960.log.gz 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53777.log.gz 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57576.log 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55762.log.gz 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47193.log 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57576.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58964.log.gz 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2024-01-19T04:58:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47193.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57576.log.gz 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47193.log.gz 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59659.log 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46384.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53819.log 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47792.log 2024-01-19T04:58:19.126 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0%/var/log/ceph/ceph-client.admin.53819.log: -- replaced with /var/log/ceph/ceph-client.admin.59659.log.gz 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60289.log 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47792.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53819.log.gz 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57802.log 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60289.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47792.log.gz 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48937.log 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60289.log.gz 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60641.log 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48937.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57802.log.gz 2024-01-19T04:58:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53861.log 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60641.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48937.log.gz 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54801.log 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60641.log.gz 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48895.log 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54801.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53861.log.gz 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59216.log 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48895.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54801.log.gz 2024-01-19T04:58:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52756.log 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48895.log.gz 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45962.log 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52756.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59216.log.gz 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42811.log 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52756.log.gz 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45962.log.gz 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55394.log 2024-01-19T04:58:19.129 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59463.log 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42811.log.gz 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59258.log 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55394.log.gz 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43287.log 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59258.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59463.log.gz 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50060.log 2024-01-19T04:58:19.130 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43287.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59258.log.gz 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48601.log 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43287.log.gz 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46426.log 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48601.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50060.log.gz -5 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51406.log 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46426.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48601.log.gz 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60247.log 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51406.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46426.log.gz 2024-01-19T04:58:19.131 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56882.log 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60247.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51406.log.gz 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54152.log 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3%/var/log/ceph/ceph-client.admin.56882.log: -- replaced with /var/log/ceph/ceph-client.admin.60247.log.gz 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46468.log 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56882.log.gz 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55566.log 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46468.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30358.log 2024-01-19T04:58:19.132 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54152.log.gz 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46468.log.gz 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55566.log.gz 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.tmp-client.admin.30358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49808.log 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30358.log.gz 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52488.log 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49808.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50578.log 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49808.log.gz 2024-01-19T04:58:19.133 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50240.log 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50578.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52488.log.gz 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56656.log 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50240.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50578.log.gz 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60331.log 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50240.log.gz 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59379.log 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60331.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56656.log.gz 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54530.log 2024-01-19T04:58:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59379.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60331.log.gz 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53945.log 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59379.log.gz 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56924.log 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53945.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54530.log.gz 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52446.log 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53945.log.gz 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53677.log 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56924.log.gz 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58838.log 2024-01-19T04:58:19.135 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53677.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52446.log.gz 2024-01-19T04:58:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51188.log 2024-01-19T04:58:19.136 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53677.log.gz 2024-01-19T04:58:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2024-01-19T04:58:19.136 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58838.log.gz 2024-01-19T04:58:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56556.log 2024-01-19T04:58:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51188.log.gz 2024-01-19T04:58:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54759.log 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9%/var/log/ceph/ceph-client.admin.56556.log: -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51842.log 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56556.log.gz 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45426.log 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54759.log: /var/log/ceph/ceph-client.admin.51842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54194.log 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54759.log.gz 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51842.log.gz 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48979.log 2024-01-19T04:58:19.137 INFO:teuthology.orchestra.run.smithi031.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45426.log.gz 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50620.log 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48979.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54194.log.gz 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49273.log 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50620.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48979.log.gz 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51230.log 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43021.log 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51230.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50620.log.gz 2024-01-19T04:58:19.138 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49273.log.gz 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52882.log 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51230.log.gz 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59421.log 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43021.log.gz 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50945.log 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59421.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52882.log.gz 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53002.log 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59421.log.gz 2024-01-19T04:58:19.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.50945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54023.log 2024-01-19T04:58:19.140 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2024-01-19T04:58:19.140 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54023.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53002.log.gz 2024-01-19T04:58:19.140 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-01-19T04:58:19.140 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54446.log 2024-01-19T04:58:19.140 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52840.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54023.log.gz 2024-01-19T04:58:19.140 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43279.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2024-01-19T04:58:19.140 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42209.log 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.54446.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36683.log 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.50945.log.gz 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44533.log 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54446.log.gz 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2%/var/log/ceph/ceph-client.admin.36683.log: -- replaced with /var/log/ceph/ceph-client.admin.42209.log.gz 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44533.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36683.log.gz 2024-01-19T04:58:19.141 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46342.log 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48087.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44533.log.gz 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42557.log 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4%/var/log/ceph/ceph-client.admin.46342.log: -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44323.log 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46342.log.gz 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43619.log 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44323.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42557.log.gz 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-01-19T04:58:19.142 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44323.log.gz 2024-01-19T04:58:19.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51448.log 2024-01-19T04:58:19.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50018.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.43619.log.gz 2024-01-19T04:58:19.143 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.57345.log 2024-01-19T04:58:19.143 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-01-19T04:58:19.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56614.log 2024-01-19T04:58:19.143 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51448.log.gz 2024-01-19T04:58:19.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.57345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43959.log 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60559.log 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43959.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56614.log.gz 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43959.log.gz 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54404.log 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.0.57345.log.gz 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60559.log.gz 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44239.log 2024-01-19T04:58:19.144 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.54404.log: /var/log/ceph/ceph-client.admin.49063.log 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46888.log 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54404.log.gz 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49063.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44239.log.gz 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49063.log.gz 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5/var/log/ceph/ceph-client.admin.46888.log: --verbose -- /var/log/ceph/ceph-client.0.58307.log 2024-01-19T04:58:19.145 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47235.log 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46888.log.gz 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.58307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51716.log 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49021.log 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47235.log.gz 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46720.log 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53467.log 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51716.log.gz 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49021.log.gz 2024-01-19T04:58:19.146 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-19T04:58:19.147 INFO:teuthology.orchestra.run.smithi031.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.0.58307.log.gz 2024-01-19T04:58:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51306.log 2024-01-19T04:58:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2024-01-19T04:58:19.147 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46720.log.gz 2024-01-19T04:58:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51306.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53467.log.gz 2024-01-19T04:58:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59785.log 2024-01-19T04:58:19.147 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51306.log.gz 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55888.log 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55888.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59785.log.gz 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50452.log 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55888.log.gz 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42979.log 2024-01-19T04:58:19.148 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43819.log 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42979.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50452.log.gz 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59911.log 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42979.log.gz 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46930.log 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43819.log.gz 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57928.log 2024-01-19T04:58:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46930.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59911.log.gz 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42465.log 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46930.log.gz 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48475.log 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57928.log.gz 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56056.log 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48475.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42465.log.gz 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42685.log 2024-01-19T04:58:19.150 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48475.log.gz 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55352.log 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56840.log 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56056.log.gz 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42685.log.gz 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59318.log 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55352.log.gz 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49105.log 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56840.log: /var/log/ceph/ceph-client.admin.59318.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56840.log.gz 2024-01-19T04:58:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47335.log 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59318.log.gz 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60415.log 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47335.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49105.log.gz 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60205.log 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47335.log.gz 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47109.log 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60415.log.gz 2024-01-19T04:58:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46300.log 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47109.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60205.log.gz 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54488.log 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47109.log.gz 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54969.log 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54488.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46300.log.gz 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59090.log 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54969.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54488.log.gz 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53903.log 2024-01-19T04:58:19.153 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54969.log.gz 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59090.log.gz 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55676.log 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50104.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53903.log.gz 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48769.log 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55676.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43105.log 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55676.log.gz 2024-01-19T04:58:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43105.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48769.log.gz 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46090.log 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43105.log.gz/var/log/ceph/ceph-client.admin.47665.log: 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55718.log 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58602.log 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55718.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46090.log.gz 2024-01-19T04:58:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42937.log 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55718.log.gz 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59006.log 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58602.log.gz 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42769.log 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59006.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42937.log.gz 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-osd.1.log 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59006.log.gz 2024-01-19T04:58:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53635.log 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42769.log.gz 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57970.log 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47377.log 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53635.log.gz 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55930.log 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60373.log 88.0% 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57970.log.gz 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47377.log: /var/log/ceph/ceph-client.admin.55930.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47377.log.gz 2024-01-19T04:58:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44659.log 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51674.log 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60373.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55930.log.gz 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53593.log 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60373.log.gz 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51674.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44659.log.gz 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54717.log 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53593.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51674.log.gz 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57702.log 2024-01-19T04:58:19.158 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53593.log.gz 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44281.log 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53551.log 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54717.log.gz 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42377.log 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57702.log.gz 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44281.log.gz 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49315.log 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53551.log.gz 2024-01-19T04:58:19.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42515.log 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42377.log.gz 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46678.log 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49315.log: /var/log/ceph/ceph-client.admin.42515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51758.log 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49315.log.gz 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1%/var/log/ceph/ceph-client.admin.46678.log: -- replaced with /var/log/ceph/ceph-client.admin.42515.log.gz 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49375.log 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51758.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46678.log.gz 2024-01-19T04:58:19.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43952.log 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51758.log.gz 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59701.log 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49375.log: /var/log/ceph/ceph-client.admin.43952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49375.log.gz 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59701.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43952.log.gz 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50368.log 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59701.log.gz 2024-01-19T04:58:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.44785.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43622.log 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46762.log 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43622.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50368.log.gz 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51632.log 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43622.log.gz 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52714.log 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51632.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46762.log.gz 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59953.log 2024-01-19T04:58:19.162 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51632.log.gz 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42025.log 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52714.log.gz 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44449.log 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59953.log.gz 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49189.log 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42025.log: /var/log/ceph/ceph-client.admin.44449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41929.log 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42025.log.gz 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44449.log.gz 2024-01-19T04:58:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56381.log 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.49189.log.gz -5 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose/var/log/ceph/ceph-client.admin.41929.log: -- /var/log/ceph/ceph-client.admin.51938.log 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.56381.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.41929.log.gz --verbose 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.58728.log 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.51938.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60457.log 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56122.log 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51938.log.gz 2024-01-19T04:58:19.164 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58728.log.gz 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58481.log 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56122.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60457.log.gz 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44701.log 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56122.log.gz 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58481.log: gzip 91.5% -- replaced with /var/log/ceph/ceph-client.0.56381.log.gz 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56698.log 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58481.log.gz 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60079.log 2024-01-19T04:58:19.165 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44701.log.gz 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52672.log 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56698.log.gz/var/log/ceph/ceph-client.admin.60079.log: 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52672.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60079.log.gz 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48559.log 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.3.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52672.log.gz 2024-01-19T04:58:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48727.log 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48559.log.gz 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44197.log 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48727.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48517.log 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44197.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48727.log.gz 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56014.log 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48517.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44197.log.gz 2024-01-19T04:58:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47623.log 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56014.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48517.log.gz 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47581.log 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47623.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56014.log.gz 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46510.log 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47581.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47623.log.gz 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54320.log 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46510.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47581.log.gz 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48045.log 2024-01-19T04:58:19.168 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46510.log.gz 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49147.log 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48045.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54320.log.gz 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46804.log 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49147.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48045.log.gz 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48685.log 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49147.log.gz 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43621.log 2024-01-19T04:58:19.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48685.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46804.log.gz 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48685.log.gz 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55522.log 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43621.log.gz 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55522.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48313.log 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43286.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55522.log.gz 2024-01-19T04:58:19.170 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55972.log 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44365.log 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50494.log 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48313.log.gz 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44365.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55972.log.gz 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51800.log 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44365.log.gz 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2024-01-19T04:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58922.log 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51800.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50494.log.gz 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54572.log 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51800.log.gz 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58922.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gzgzip 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56782.log 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54572.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58922.log.gz 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59995.log 2024-01-19T04:58:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54236.log 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54572.log.gz 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59995.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56782.log.gz 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59995.log.gz 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49766.log 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48171.log 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54236.log.gz 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42895.log 2024-01-19T04:58:19.173 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49766.log.gz 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48171.log.gz 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59048.log 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51490.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42895.log.gz 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46258.log 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48355.log 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46258.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59048.log.gz 2024-01-19T04:58:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55011.log 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46258.log.gz 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48355.log: /var/log/ceph/ceph-client.admin.47749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47067.log 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48355.log.gz 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55011.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58560.log 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55011.log.gz 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43063.log 2024-01-19T04:58:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47151.log 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58560.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47067.log.gz 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36802.log 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58560.log.gz 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47151.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43063.log.gz 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57744.log 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47151.log.gz 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58048.log 2024-01-19T04:58:19.176 INFO:teuthology.orchestra.run.smithi031.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36802.log.gz 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57744.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58048.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57744.log.gz 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50148.log 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mgr.y.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58048.log.gz 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42148.log 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59505.log 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42148.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50148.log.gz 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36429.log 2024-01-19T04:58:19.177 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42148.log.gz 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55846.log 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59505.log.gz 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47837.log 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55846.log: 92.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42727.log 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36429.log.gz 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55846.log.gz 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49418.log 2024-01-19T04:58:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.42727.log: 89.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.59174.log 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47837.log.gz 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49418.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44407.log 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42727.log.gz 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49418.log.gz 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52614.log 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59174.log: /var/log/ceph/ceph-client.admin.44407.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59174.log.gz 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.42293.log 2024-01-19T04:58:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52614.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44407.log.gz 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43615.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52614.log.gz 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42293.log: gzip -5 --verbose 89.8% -- /var/log/ceph/ceph-client.admin.60037.log 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42293.log.gz 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50198.log 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60037.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43615.log.gz 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2024-01-19T04:58:19.180 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60037.log.gz 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48003.log 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50198.log.gz 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58880.log 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48003.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48129.log 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48003.log.gz 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52197.log 2024-01-19T04:58:19.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59827.log 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48129.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58880.log.gz 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.52197.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48129.log.gz 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43623.log 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59743.log 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59827.log.gz 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58644.log 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43623.log.gz 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59743.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.52197.log.gz 2024-01-19T04:58:19.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59869.log 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59743.log.gz 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51574.log 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58644.log: /var/log/ceph/ceph-client.admin.59869.log: gzip 87.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.43282.log 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58644.log.gz 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59869.log.gz 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57518.log 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51574.log.gz 2024-01-19T04:58:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43282.log: /var/log/ceph/ceph-client.admin.57518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55616.log 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43282.log.gz 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57518.log.gz 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45798.log 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44617.log 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55616.log: /var/log/ceph/ceph-client.2.45798.log: gzip -5 --verbose 88.5% -- /var/log/ceph/ceph-client.admin.48643.log 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55616.log.gz 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58686.log 2024-01-19T04:58:19.184 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2%/var/log/ceph/ceph-client.admin.48643.log: -- replaced with /var/log/ceph/ceph-client.admin.44617.log.gz 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.2.45798.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48643.log.gz 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55148.log 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58686.log: /var/log/ceph/ceph-client.admin.52572.log: gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58686.log.gz 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.56966.log 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55148.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52798.log 2024-01-19T04:58:19.185 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55148.log.gz 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43958.log 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56966.log.gz 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57008.log 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43958.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52798.log.gz 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57886.log 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43958.log.gz 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44155.log 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57886.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57008.log.gz 2024-01-19T04:58:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53261.log 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57886.log.gz 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43954.log 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44155.log: /var/log/ceph/ceph-client.0.53261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44155.log.gz 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44491.log 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43954.log: /var/log/ceph/ceph-client.admin.49850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42853.log 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44491.log: 88.6% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2024-01-19T04:58:19.187 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43954.log.gz 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44491.log.gz 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.53261.log.gz 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54614.log 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42853.log.gz 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48397.log 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54614.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-19T04:58:19.188 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54614.log.gz 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1%/var/log/ceph/ceph.log: -- replaced with /var/log/ceph/ceph-client.admin.48397.log.gz 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45304.log 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49231.log 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.45304.log: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57844.log 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48853.log 2024-01-19T04:58:19.189 INFO:teuthology.orchestra.run.smithi031.stderr: 90.3% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49231.log.gz 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.0.45304.log.gz 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57844.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57844.log.gz 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43285.log 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8%/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.48853.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55478.log 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42251.log 2024-01-19T04:58:19.190 INFO:teuthology.orchestra.run.smithi031.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43285.log.gz 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.55478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60599.log 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0%/var/log/ceph/ceph-client.admin.42251.log: -- replaced with /var/log/ceph/ceph-client.admin.55478.log.gz 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42251.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46846.log 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57086.log 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46846.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54843.log 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60599.log.gz 2024-01-19T04:58:19.191 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46846.log.gz 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55310.log 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44743.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57086.log.gz 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54843.log.gz 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54278.log 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5/var/log/ceph/ceph-client.admin.55310.log: --verbose -- /var/log/ceph/ceph-client.admin.49934.log 2024-01-19T04:58:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45920.log 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55310.log.gz 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54278.log.gz/var/log/ceph/ceph-client.admin.49934.log: 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52404.log 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49934.log.gz 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47707.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43957.log 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45920.log.gz 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-01-19T04:58:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49460.log 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55053.log 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52404.log.gz 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49460.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43957.log.gz 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48271.log 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49460.log.gz 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5/var/log/ceph/ceph-client.admin.55053.log: --verbose -- /var/log/ceph/ceph-client.admin.46594.log 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48271.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55053.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54885.log 2024-01-19T04:58:19.194 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48271.log.gz 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46594.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46594.log.gz 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54674.log 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54885.log: -- /var/log/ceph/ceph-client.admin.55804.log 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54674.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54885.log.gzgzip 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54674.log.gz 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-01-19T04:58:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55804.log: /var/log/ceph/ceph-client.admin.46174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42421.log 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46636.log: 88.1% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55804.log.gz 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42421.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42421.log.gz 2024-01-19T04:58:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49976.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47419.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57660.log 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43611.log 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7%/var/log/ceph/ceph-client.admin.57660.log: -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57660.log.gz 2024-01-19T04:58:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log 2024-01-19T04:58:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43611.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43611.log.gz 2024-01-19T04:58:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53735.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2024-01-19T04:58:19.199 INFO:teuthology.orchestra.run.smithi031.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-19T04:58:19.199 INFO:teuthology.orchestra.run.smithi031.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-19T04:58:19.199 INFO:teuthology.orchestra.run.smithi137.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-19T04:58:19.199 INFO:teuthology.orchestra.run.smithi137.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-19T04:58:19.199 INFO:teuthology.orchestra.run.smithi137.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-19T04:58:19.200 INFO:teuthology.orchestra.run.smithi137.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-19T04:58:19.200 INFO:teuthology.orchestra.run.smithi137.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-19T04:58:19.207 INFO:teuthology.orchestra.run.smithi137.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-19T04:58:19.209 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-01-19T04:58:19.209 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m0.225s 2024-01-19T04:58:19.209 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m1.014s 2024-01-19T04:58:19.209 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m0.052s 2024-01-19T04:58:19.219 INFO:teuthology.orchestra.run.smithi031.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-19T04:58:19.229 INFO:teuthology.orchestra.run.smithi031.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-19T04:58:19.268 INFO:teuthology.orchestra.run.smithi031.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-19T04:58:19.273 INFO:teuthology.orchestra.run.smithi031.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-19T04:58:19.275 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-19T04:58:19.275 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m0.317s 2024-01-19T04:58:19.275 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m1.214s 2024-01-19T04:58:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.254s 2024-01-19T04:58:19.276 INFO:tasks.ceph:Archiving logs... 2024-01-19T04:58:19.276 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/remote/smithi031/log 2024-01-19T04:58:19.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-19T04:58:19.753 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/remote/smithi137/log 2024-01-19T04:58:19.754 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-19T04:58:19.910 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-19T04:58:19.925 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-19T04:58:19.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-19T04:58:19.928 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-19T04:58:20.008 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-19T04:58:20.017 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-19T04:58:20.017 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-19T04:58:20.020 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-19T04:58:20.032 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-01-19T04:58:20.033 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-01-19T04:58:20.036 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-19T04:58:20.037 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-01-19T04:58:20.037 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +288us[ +288us] +/- 95ms 2024-01-19T04:58:20.037 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -171us[ -164us] +/- 58ms 2024-01-19T04:58:20.037 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +9625ns[ +17us] +/- 73ms 2024-01-19T04:58:20.037 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:58:20.039 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-19T04:58:20.039 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-01-19T04:58:20.039 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 +293us[ +304us] +/- 95ms 2024-01-19T04:58:20.039 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -161us[ -151us] +/- 58ms 2024-01-19T04:58:20.039 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +6712ns[ +17us] +/- 73ms 2024-01-19T04:58:20.039 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-19T04:58:20.040 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-19T04:58:20.049 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-19T04:58:20.049 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-19T04:58:20.058 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-19T04:58:20.156 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-19T04:58:20.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-19T04:58:20.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:58:20.253 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-19T04:58:20.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:58:20.282 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-19T04:58:20.292 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-19T04:58:20.312 INFO:teuthology.task.internal:Duration was 589.265648 seconds 2024-01-19T04:58:20.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-19T04:58:20.321 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-19T04:58:20.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-19T04:58:20.324 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-19T04:58:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-19T04:58:20.367 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-19T04:58:20.773 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-19T04:58:20.773 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-01-19T04:58:20.773 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-19T04:58:20.833 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-01-19T04:58:20.833 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-19T04:58:20.855 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-19T04:58:20.855 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-19T04:58:20.876 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-19T04:58:20.924 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-19T04:58:20.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-19T04:58:20.945 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-19T04:58:21.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-19T04:58:21.070 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-19T04:58:21.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-19T04:58:21.097 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-19T04:58:21.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-19T04:58:21.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-19T04:58:21.140 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-19T04:58:21.162 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-01-19T04:58:21.189 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-01-19T04:58:21.205 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-19T04:58:21.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:58:21.232 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-19T04:58:21.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-19T04:58:21.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-19T04:58:21.270 INFO:teuthology.task.internal:Transferring archived files... 2024-01-19T04:58:21.270 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/remote/smithi031 2024-01-19T04:58:21.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-19T04:58:21.308 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038/remote/smithi137 2024-01-19T04:58:21.308 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-19T04:58:21.344 INFO:teuthology.task.internal:Removing archive directory... 2024-01-19T04:58:21.344 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-19T04:58:21.346 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-19T04:58:21.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-19T04:58:21.403 INFO:teuthology.task.internal:Not uploading archives. 2024-01-19T04:58:21.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-19T04:58:21.415 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-19T04:58:21.416 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-19T04:58:21.418 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-19T04:58:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: 265291 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 19 04:58 /home/ubuntu/cephtest 2024-01-19T04:58:21.446 INFO:teuthology.orchestra.run.smithi137.stdout: 266028 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 19 04:58 /home/ubuntu/cephtest 2024-01-19T04:58:21.447 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-19T04:58:21.458 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-19T04:58:21.479 INFO:teuthology.nuke:Checking targets against current locks 2024-01-19T04:58:21.505 DEBUG:teuthology.nuke:shortname: smithi031 2024-01-19T04:58:21.505 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-19T04:58:21.529 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-19 04:40:25.451500', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfCsrSuIklKEIe2LJNcxUOdkdiyQIl89QsL9zj3TrLRIJqWDvPNGcqZ4/uam4JLHEob6pP0RSQxbCMD2dg06FQ='} 2024-01-19T04:58:21.532 DEBUG:teuthology.nuke:shortname: smithi137 2024-01-19T04:58:21.533 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-19T04:58:21.560 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-18_21:19:14-fs-wip-yuri8-testing-2024-01-18-0823-distro-default-smithi/7521038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-19 04:40:25.450179', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNazwqQDaviTxfMbB+N/FPtO/PTb6ysR06SdlXD5H6640Cm2XYEZznkQmi/ZEo90JW1vpWugaybplenJ7ZsRFEw='} 2024-01-19T04:58:21.585 INFO:teuthology.orchestra.console.smithi031:Power off 2024-01-19T04:58:21.585 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-19T04:58:21.608 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2024-01-19T04:58:21.608 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:21.626 INFO:teuthology.orchestra.console.smithi137:Power off 2024-01-19T04:58:21.627 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-19T04:58:21.636 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-01-19T04:58:21.652 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-01-19T04:58:21.652 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:21.677 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-01-19T04:58:25.637 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:25.678 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:25.748 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-01-19T04:58:25.789 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-01-19T04:58:29.752 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:29.790 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:29.863 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-01-19T04:58:29.899 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-01-19T04:58:33.866 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:33.899 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-19T04:58:33.977 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-01-19T04:58:34.010 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-01-19T04:58:34.079 INFO:teuthology.orchestra.console.smithi031:Power off completed 2024-01-19T04:58:34.111 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-01-19T04:58:34.216 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm} duration: 589.2656478881836 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c00d74b8f14945ddbe0d34c44d48a4ab status: fail success: false 2024-01-19T04:58:34.217 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-19T04:58:34.294 INFO:teuthology.run:FAIL